Skip to content
← All articles

Runway Gen-2 tests: a rainforest in 4-second clips (2023)

Lab note · Tested July 2023 · · 3 min read

An AI-generated jaguar standing in turquoise water in a green rainforest, a frame from one of our Runway Gen-2 test clips

In late July and early August 2023 we tested Runway Gen-2, one of the first tools that could turn a picture or a sentence into a moving clip. We share the test because it shows clearly where the craft of AI video started.

One world, 32 clips

Could an AI tool give us usable shots for one consistent story? We picked a single world, a rainforest, and tried to build a sequence of shots inside it:

  • wide shots: a river valley from above, storm clouds over water, Earth from space
  • creatures: a jaguar in shallow water, butterflies in shafts of light
  • characters: a forest spirit covered in leaves, an explorer team, a man with face paint and a headband
  • details: a close-up of an eye, a kayak moving down a jungle river

We made 32 clips. Every clip is 4 seconds long, at 896 × 512 pixels. Several shots exist in two to four versions, because we generated the same idea again and again to get one usable take.

The look is strong. The clips have rich colour, soft light and a cinematic mood, which was new for AI in 2023.

Where the faces drift

The movement is the weak point. In most clips the camera slowly pushes in or drifts, and the subject moves very little. When a subject does move, it changes. In the explorer clip below, the woman in the first frame and the woman in the last frame are not quite the same person: the face, the hair and the people behind her all shift within 4 seconds.

The first frame of the explorer clip: a young woman with blond hair in a forest, with a man on her left and a woman behind her
First frame, at 0 seconds: a young woman with blond hair stands in a forest with two companions.
The last frame of the same clip: her face is different and out of focus, and a person in a red jacket stands behind her
Last frame, at 4 seconds: her face and the people behind her have changed. The frames are the clip's own size, 896 × 512 pixels.

What we learned

  1. Short clips, many takes. At 4 seconds a clip, a story is built from many cuts. Plan the edit first, then generate shots to fit it.
  2. Faces drift. Keeping one character the same from shot to shot was the hardest part. In 2023 the only fix was to cut before the drift became visible.
  3. Mood is easy, action is hard. Landscapes, weather and slow camera moves worked well. A person or an animal doing something specific did not.
  4. Name your files. Runway saved each clip under a number, not the prompt. Today we cannot say which words made which clip, so we cannot rerun the exact test on a newer tool.

What still holds

Current video models hold a face and follow an action far better than this. The working method has not changed, though: decide the story and the edit first, generate several takes of each shot, pick the best, and keep the prompt with every file. That method is what we teach in our workshops, and it is the same one a film crew uses.

For a test where we planned and documented everything, read about our Runway Gen:48 short film.

More to read

Talk to us