What to know

  • Meta has unveiled its new AI model – Movie Gen, that generates video from text inputs, similar to OpenAI’s yet unreleased Sora. 
  • Apart from generating videos, Movie Gen can also create videos from photos, edit videos using text inputs, and generate audio for videos. 
  • Movie Gen is still a work-in-progress. A public release hasn’t been announced.  

Meta’s Movie Gen looks nothing less than magic. With a simple text prompt, the company’s new AI model can generate video clips, edit existing ones, turn your photos into videos, and even generate audio – a complete creator’s package.

Similar to other text-to-video models, like OpenAI’s Sora, Movie Gen isn’t available for use yet, and everything that Meta has demonstrated is for a future release. But the basic idea is the same – input text and generate high-definition videos.

Videos can be generated in different aspect ratios too.

On top of that, Movie Gen also works as a video editor, allowing you to upload a video and use prompts to make precise cuts, add styles and transitions, and transform them with AI.  

One of the standout features in Movie Gen is ‘Personalized videos‘, a feature that uses photos to create videos while preserving human identity and expressions.

The model can also generate audio for a video (with or without text prompts) for up to 45 seconds long. This includes sound effects, instrumental music, and ambient sound that is synced to the video. There’s also an audio extension technique being used to ensure that the generated audio coheres with the video. 

Meta believes the model “could accelerate creativity” but has also said that it isn’t “ready to release this as a product anytime soon – it’s still expensive and generation time is long – but we wanted to share were we are since the results are getting quite impressive.” Although AI video generators are still a thing of the future, Movie Gen puts Meta in competition with companies like OpenAI and Google that are developing their own text-to-video AI models. 

As we’ve learned from AI image generators, such powerful AI video generators have high potential for misuse. It could eventually pose a threat to the livelihood of filmmakers and video creators. It may not be long before completely AI-generated movies start infiltrating the market. But the technology is still a work in progress. Meta’s given no tentative release dates but we don’t expect it to arrive anytime before late next year.