OpenAI's text-to-video model is blowing people's minds

OpenAI’s new AI-made videos are blowing people’s minds

An AI image portraying two mammoths that walk through snow, with mountains and a forest in the background. — OpenAI

OpenAI’s latest venture into AI might be its most impressive one to date. Dubbed “Sora,” this new text-to-video AI model has just opened its doors to a limited number of users who will get to test it. The company launched it by showing several videos made entirely by AI, and the end results are shockingly realistic.

OpenAI introduces Sora by saying that it can create realistic scenes based on text prompts, and the videos shared on its website serve to prove it. The prompts are descriptive, but short; I’ve personally used longer prompts just interacting with ChatGPT. For instance, to generate the video of wooly mammoths pictured above, Sora required a 67-word prompt that described the animals, the surroundings, and the camera placement.

Introducing Sora, our text-to-video model.

Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. https://t.co/7j2JN27M3W

Prompt: “Beautiful, snowy… pic.twitter.com/ruTEWn87vf

— OpenAI (@OpenAI) February 15, 2024

“Sora can generate videos up to a minute long while maintaining visual quality and adherence to the user’s prompt,” said OpenAI in its announcement. The AI can generate complex scenes filled with many characters, scenery, and accurate motion. To that end, OpenAI says that Sora predicts and reads between the lines as needed.

“The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world,” OpenAI said. The model doesn’t just tackle characters, clothing, or backgrounds, but also creates “compelling characters that express vibrant emotions.”

Sora can also fill in the gaps in an existing video or make it longer, as well as generate a video based on an image, so it’s not all just text prompts.

While the videos look good as screenshotted stills, they’re borderline mind-blowing in motion. OpenAI served up a wide range of videos to show off the new tech, including Cyberpunk-esque Tokyo streets and “historical footage” of California during the Gold Rush. There’s more, too, including an extreme close-up of a human eye. The prompts cover anything from cartoons to wildlife photography.

Sora still made some mistakes. Looking closer reveals that, for instance, some figures out in the crowd don’t have heads or move strangely. The awkward motion stood out at first glance in some samples, but the general weirdness took multiple viewings to spot.

It might be a while before OpenAI opens Sora to the general public. Right now, the model will be tested by red teamers who will assess potential risks. Some creators will also get to start testing it now, while it’s still in the early stages of development.

AI is still imperfect, so I went in expecting something quite messy. Whether it’s the low expectations or Sora’s capabilities, I’m walking away impressed, but also mildly worried. We’re already living in a world where it’s hard to tell a fake from something real, and now, it’s not just images that are in jeopardy — videos are, too. However, Sora is hardly the first text-to-video model we’ve seen, such as Pika.

Others are raising the flag as well, such as the popular tech YouTuber, Marques Brownlee, who tweeted that “if this doesn’t concern you at least a little bit, nothing will” in response to the Sora videos.

Every single one of these videos is AI-generated, and if this doesn't concern you at least a little bit, nothing will

The newest model: https://t.co/zkDWU8Be9S

(Remember Will Smith eating spaghetti? I have so many questions) pic.twitter.com/TQ44wvNlQw

— Marques Brownlee (@MKBHD) February 15, 2024

If OpenAI’s Sora is this good now, it’s hard to imagine what it’ll be capable of after a few years of further development and testing. This is the kind of tech that has the potential to displace many jobs — but, hopefully, like ChatGPT, it will instead coexist alongside human professionals.

Editors' Recommendations