Unleash Your Creative Power with Runway AI Video Generation: A Comprehensive Guide

Generative AI is garnering significant attention these days, and in this article, we'll explore the features and usage of Runway, a video generation AI platform.

For an overview, please check out this article.

What is Runway?

Runway is the name of a video generation AI platform.

It's accessible on both smartphones and PCs, allowing you to create new video data from existing video footage or generate videos from scratch using text prompts.

This article will primarily focus on the features and operation methods available on the PC version.

Runway's Core Features

Runway's basic features mainly consist of four components:

• Video generation/editing features

• Image generation/editing features

• 3D model generation features

• Custom learning model building features

In this article, we will focus on video generation, so we will elaborate on the video generation and editing features.

Video Generation Feature: Gen-1

First, let's discuss Gen-1, which allows you to generate new videos based on existing video data.

As mentioned above, Gen-1 generates new videos by inputting arbitrary prompts or uploading additional images to the uploaded video data.

Since the initially uploaded video serves as the base, the generated video is more of a variation than a completely new creation.

Gen-1 is ideal for situations where you want to expand on ideas by adding various variations to a concept that has already taken shape, rather than creating something from scratch.

Video Generation Feature: Gen-2

Next up is Gen-2.

Unlike Gen-1, Gen-2 can generate entirely new videos without existing video data.

Specifically, you can input your desired video image as a prompt or upload an image file to generate a video from scratch.

Gen-2 is valuable in the early stages of the video creation process, such as when you're brainstorming video ideas. It proves most useful in the initial planning phases.

Video Editing Features

Finally, let's explore the video editing features.

As mentioned at the beginning of this article, Runway is equipped not only with video generation but also with editing features.

Here's a list of specific features:

Remove Background: Background Removal

As the name suggests, this feature removes the background of uploaded video data.

After uploading the video data, you select the parts to be removed and the parts to be kept, and the video is edited accordingly.

Although it doesn't automatically detect the background, the ability to specify the areas you want to keep makes it easy to use, in my opinion.

[Before]

[After]

*In this case, I set the deleted part to be a green screen, resulting in this type of video.

Inpainting: Removing Part of a Video

While Remove Background "recognizes and removes everything except the specified parts as the background," Inpainting is a "feature to remove the specified parts."

[Before]

[After]

I tried removing the figure in the same video.

(The video didn't change much, so I uploaded it as an image.)

Color Grade: Adjusting Colors

Color Grade is a feature that allows you to adjust the color tones of a video.

Rather than selecting from a set of predefined patterns, it generates videos by specifying colors with prompts.

This time, I specified "mid blue" for the Before video.

[Before]

[After]

Super-Slow motion: Slow Motion Editing

This feature allows you to edit the uploaded video into slow motion.

You can specify the desired degree of slowness for the edit.

Blur Faces: Preventing Portrait Rights Infringement

This feature automatically detects faces in the uploaded video and applies corrections (mosaic or blur) to them.

It can be used to protect the portrait rights and privacy of people who happen to be captured in the original video used in Gen-1.

Depth of Field: Blurring Feature

This feature adds blur to the uploaded video.

[Before]

[After]

Scene Detection: Automatic Cut Point Detection

This feature is intended for editors. Instead of treating the video as one work, it automatically detects the points where the video should be divided into scenes (cut points) to facilitate editing.

For those involved in professional video production, the automatic detection of cut points without manual setting may be a welcome feature.

Extract Depth: Automatic Depth Map Generation

This feature, which may not be used by many, automatically generates a depth map (an image that converts the relative distance between the shooting position and the object being shot into color intensity).

It may be a little difficult to understand in words, so please check the image below.

[Before]

[After]

Motion Tracking: Automatic Object Tracking

This feature allows you to specify an object in the uploaded video and generate a video that automatically tracks that object.

*The person in the video I created was too small to be considered an object, so it seems that it is necessary for the object to be of a certain size.

We have now covered the list of features. Next, let's actually use Runway.

How to Use Runway (Gen-2)

In this article, we will mainly introduce how to use Gen-2 (PC version), which many people will probably try.

We will divide the explanation into preparation and practice sections.

[Preparation] Search for "Runway AI"

First, search for Runway AI in any web browser.

You should see a web page like the attached image in the search results, so click on it.

[Preparation] Log in with a Google account or create an account with your email address and log in

Once you have successfully accessed the attached image, you will see "Login/Sign up" in the upper right corner, so if you can log in with your Google account, Apple account, or other SSO, press the Login button.

For those who do not have an account, press the Sign up button to create an account first.

That's it for the preparation.

Let's start generating videos with Gen-2.

[Practice] Launch Gen-2

After successfully logging in, you will see the top page as shown in the attached image, so press Generate Videos from the menu bar on the left side of the screen.

When you select it, the screen on the right will change, and an icon labeled "Gen-2" will be displayed near the bottom center of the screen, so select it.

[Practice] Enter a Prompt in Gen-2 and Generate a Video

Once Gen-2 is launched, you will see the following screen.

To briefly explain each tab:

• Text -> Generates video from the entered text (prompt)

• Image -> Generates video from the uploaded image

• Image + Description -> Generates a video based on the uploaded image, specifying more detailed conditions and how to move the image as a video using prompts

Let's use it right away.

I selected Image + Description and specified "the sun sets from right to left" for an image of Mount Fuji.

[Before]

[After]

The entered prompt may not have been appropriate due to my English skills, but I think it came out pretty close to what I imagined.

[Practice] Edit the Generated Video

Finally, let's edit the generated video.

Select Edit Videos from the menu bar on the left side of the top page and select your favorite editing function.

I wanted to make the generated video a little more stylish, so I selected Depth of Field, which can add blur.

The editing function has a simpler specification compared to video generation, so I think it's easy to use.

[Before]

[After]

I feel like it's become a little harder to see rather than stylish, but let's just say that's also a pleasure.

In this way, anyone can easily generate and edit videos, so please try various things.

Tips for Using Runway

Here are three tips for using Runway:

Keep Trying Until You Can Generate the Ideal Video

The first is to keep trying until you can generate the ideal video.

This can be said in general for generative AI, but since AI is not a person, it does not mean that the ideal thing will be created in one try.

Try various things, such as changing the prompt input, changing the image to upload, and making good use of the editing functions, to generate the ideal video.

Enter the Prompt in Sentences as Much as Possible

The second is to enter the prompt in sentences as much as possible.

When creating videos and content, the ease of creation as a creator also changes depending on how much information the client has about the ideal image, but the same is true for AI.

Rather than arranging two or three words in the prompt, it is easier to generate a video that matches the image if you describe more detailed conditions in sentences.

Use Google Translate etc. to enter it in sentences.

Make Use of Functions such as RemoveBackground

The third is to make use of functions such as RemoveBackground.

The feature of Runway is its rich set of implemented functions.

Please make use of it when what is generated is a little different from the image or when you want different variations.

Runway Pricing Plans

Finally, here is an introduction to pricing plans for those who want to use Runway in earnest.

The price range is wide, ranging from about 2,000 yen/month to 10,000 yen/month (as of December 2023).

Please select the plan that you need depending on your usage.

Conclusion

In this article, we talked about how to use Runway, from the basics to the content including actual usage.

I hope this article will help you make use of generative AI, including Runway. Thank you for reading to the end.