Luma Labs recently introduced a new AI-powered video generator called the Dream Machine. Dream Machine allows anyone to easily create cinematic style videos from images and text prompts.

In initial reviews, users have found that while tools like OpenAI’s Sora can create more realistic looking videos, the Dream Machine is fast, capable of generating 120 frames in just 120 seconds. Its free tier comes with 3 generations per day and 30 generations per month, plus both text to video, image to video, and video extension capabilities.

Dream Machine is an exciting step towards enabling people to make video clips without needing to capture footage. Below, I’ll explain how to get access to the Dream Machine and use it to make videos. 

What is the Dream Machine? 

The Dream Machine uses AI to create 5-second video clips from text and images. The 5-second clips can be edited together to make brand videos, social media content, or personal projects.  

Dream Machine works on both mobile and desktop web with a simple interface. To get access, head to the Luma Labs website and sign in with a Google Account. At the time of writing, Luma Labs has limited usage to 3 generations per day for free users.

Dream Machine offers the ability to create 30 free videos per month. But, if you’re a creator who plans to use the tool more extensively, you may consider upgrading to a paid plans, which offer more features, faster processing, and higher usage limits.

Dream Machine subscription prices

• Standard: 120 generations, $29.99 a month
• Pro: 400 generations, $99.99 a month
• Premier: 2,000 generations, $499.99 a month

How to use Dream Machine’s text to video generator

Dream Machine has a text input box where you can generate a video using AI for free and a toggle for "Enhancing" or transforming the input prompt. Read through the prompt tips and type your prompt into the search box. With the right prompt, Dream Machine can generate interesting camera motions and angles, not just static shots, adding dynamism to the videos.

Once you’ve clicked "Try now," created an account and logged in, you will see the prompt input box (screenshot below). Follow these steps to generate a video:

1. Input the text you want to turn into a video. This could be a snippet from a script, a blog post, or any written content.
2. Click the Generate arrow and let the AI work its magic. Your video will be ready within a few seconds (barring any processing delays).
3. Preview the video. On Dream Machine, you can "Extend" a video, making it longer using generative AI by adding another clip to the end.
4. Download to share. You generations are accessible from the Gallery.

Although Dream Machine can give you beautiful video clips, you'll need a general purpose video editor to edit and combine the clips together into a longer-form video or collage. Kapwing allow you to montage your generated clips together, trim down the generated videos to shorter lengths, add accurate text, and add music. You can even generate AI voices for the AI videos you've edited, matching the generated videos with synthetic voices.

Extending a Video in Dream Machine

Once you've generated a video, use the "Extend" button to add another 5 seconds. This can create a longer video clip with character consistency.

Below is an example of a 10-second video made from a 5-second video of "a person editing a video." Although the original clip did not show the editor's face, I used the Extend feature to make the clip longer with the prompt "She turns off her computer and stands up."

When I extend and specify the subject in the new prompt ("she"), the video zooms out and captures the typist's face. Although the video's subject does not "stand up" or "turn off the computer," it's a good demonstration of Dream Machine's capabilities.

How to use Dream Machine’s image to video generator

Dream Machine also supports transforming an image into a Video using AI and animation between two images.

1. Select the image that you want to use as the first frame of the generated video. To upload it, click the picture icon on the left side of the text input. You can upload a second image to define the start and end frame; Dream Machine will interpolate between the two images. Type a prompt (optional) to define the action you want in the video.

2. Click on the Generate arrow to create your video.
3. In the "My Creations" gallery, choose the video you want to download and click the "Download" button to save the video to your computer or phone.

Here's an example video that we generated from a hand drawn image of Ariana Grande:

Video Specs

• File Type: Dream Machine downloads are MP4 files.
• Duration: Dream Machine clips are 5 seconds long. Extend increases the duration by 5 seconds each time it is used.
• Resolution: Dream Machine can generate videos at 1024x1024 pixel resolution, which is typically higher than the quality of similar AI Video Generators.
• Size: Downloaded files were between 2-5MB.
• Speed: The promise of the tool is that it can create 120 frames in 120 seconds. However, initial users have reported it taking upwards of 40 minutes to process their video due to high volume on the site. It appears that Dream Machine slows down when a high volume of people are using it.
• Licensing: Videos created with Dream Machine come with a standard license, allowing personal and commercial use. However, it’s always smart to review the specific licensing terms provided by Luma Labs to ensure you’re compliant. 

What Data is Dream Machine Trained on?

While Runway ML uses various pre-trained models for different tasks, Dream Machine is trained directly on video data, allowing it to better understand and generate video content. Although Luma Labs has not disclosed its training set, the model can generate various camera motions and angles, suggesting that the training data included videos with diverse cinematography techniques.

Limitations: common issues and best practices

Dream Machine has been trained to understand and generate realistic motion, actions, and physics in videos. It's ideal for generating generic realistic scenes, like "A woman drinking a coffee." However, it still often misunderstands motion and produces in-coherent results. Here are some issues that I ran into while testing Dream Machine:

• Processing time: Text-to-video prompts typically process faster than image-to-video. Some users have reported that using image prompts resulted in their video taking 45 minutes to process.

• Morphing Movement: Users have reported issues like morphing meaning that objects change form during a complicated action, resulting in unnatural looking movements.

• Misrepresenting real people: In our testing, we saw that Dream Machine did a poor job of representing real people, like celebrities or world figures. The faces and likeness was inaccurate.
• Anatomy: Often, faces, noses, limbs, and other anatomical features of the video's subjects are inaccurate. Just like AI Image generators, AI Video Generators struggle to accurately render hands and faces, the most complex anatomical structures with  intricate details, subtle movements, and precise proportions. These abnormalities make the videos uncanny.

• Compatibility: Ensure your browser is updated with the latest version available for the most optimal performance while using the Dream Machine.
• Audio: Dream Machine videos are mute, with no audio stream or sound effects.

While Luma’s Dream Machine has some bugs to work out, the biggest draw to this tool is that it has a pretty robust free tier. It's comparable to Runway and Pika in terms of it's quality, and it's completely free to try. Other AI video generators are not as accessible: Sora AI is not currently available to the public (while it’s being safety tested) and Kling AI is only available in China. For creators who want to experiment with AI video, the Dream Machine is a good place to start.