Movies are an art form that combines sound, image, plot, acting, and other elements. The production process usually requires a lot of human, material, and financial resources. But with the development of artificial intelligence (AI) technology, especially OpenAI’s latest “text-to-video” technology Sora, this may all change.

Sora can generate rich and realistic videos based on simple text prompts, and even create shots and switch scenes by itself, allowing ordinary people to make their own movies. The technology has drawn wide attention, with some saying “reality will cease to exist”.

On X (formerly Twitter), some even argued that “in the future, you only need to input a script, and the movie can be made. You no longer need dozens or hundreds of people to shoot it, nor do you need any photography team, actor team, costume team, makeup team, lighting team, driver team, etc. You don’t need to manage the crew, you just need to master the relevant knowledge, and then use AI… The future without asking for help is coming soon.”

According to a director , tools like Sora will greatly improve the efficiency of movie production and solve the problem of mismatch between the pre- and post-production stages. The emergence of such technology may indeed replace the costumes, makeup, and props (referred to as “costume makeup props”) of movies to some extent, but there are also difficulties.

As for the quality of the videos generated by Sora and other AI tools in the actual movie production process, they still hold a conservative attitude. One director told NBD bluntly that some scenes that require real actors to express huge emotional tension, if replaced by AI-generated images, the audience would have a hard time empathizing or even have a repulsive emotion.

Photo/ Screenshot from Sora’s official website demo video

Sora represents a huge leap in “text-to-video” technology

NBD noticed that most major AI companies have been researching “text-to-video” tools for years, and some companies have already released software that users can use. Before Sora was released, the latest version of Runway was the most advanced “text-to-video” technology, which can produce short videos based on text prompts.

New York Magazine reported that Sora is not yet available to ordinary users. But according to the videos generated by Sora based on text prompts released by OpenAI, it seems to be a significant technological leap in terms of realism, timely response, video length, and functionality.

In terms of details, Sora’s demo video has smooth camera movement and scene transitions, and can also create shots and switch scenes by itself, which means that Sora may allow ordinary people to make their own movies. What’s more noteworthy is that Sora’s video, after switching the perspective and scene, still maintains the consistency of the subject.

OpenAI’s official website introduces, “Sora can generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. The model can not only understand the user’s requirements in the prompt, but also understand how these things exist in the real world.” This means that if GPT can understand the world through human language, then Sora can understand the world through multimodal data such as video and image.

The Hollywood reported that although similar “text-to-video” tools have appeared before, Sora indeed represents the rapid development of this technology, which may lead to a lot of layoffs.

A survey released in January this year showed that three-quarters of the respondents in Hollywood expected AI will cause their companies to lay off or merge. It is estimated that nearly 204,000 similar jobs will be affected by AI in the next three years. According to the survey, sound engineers, voice actors, and concept artists will be the first to bear the brunt, and visual effects and other post-production will also be affected. If the technology continues to advance, these jobs will be increasingly replaced by AI tools.

Tools like Sora may replace “costume makeup props”, but there are still difficulties

Sora’s breakthrough progress has made the whole Hollywood nervous. Will Sora really disrupt the movie industry, and will the future movie shooting no longer need “costume makeup props” and other staff?

On this issue, Jiang Lai, a director and screenwriter signed by Jiangjie Universe (Beijing) Entertainment Technology Co., Ltd., and the director and screenwriter of the sci-fi hero-themed web series “Jiangjie”, told NBD that “(tools like Sora’s ‘text-to-video’ tool) may replace ‘costume makeup props’, but this may take some time, and this time may not be short. I’m not sure if Sora’s technology can really achieve the kind of final product effect of future movies or TV shows.”

In Jiang Lai’s view, the movie industry’s attitude towards tools like Sora should be divided into two camps: the first camp may have a natural repulsion to AI-generated videos because they feel fake. The second camp will enthusiastically embrace tools like Sora, and keep up with the latest status, or even try it out or find ways to use it as a tool to better carry out movie creation.

Andy, the founder of Sichuan 28th Street Cultural Communication Co., Ltd., an independent film producer, also told NBD that tools like Sora may indeed replace the “costume makeup props” of movies to some extent, but there are also difficulties. First of all, it may be very complicated for people to describe the movie scene with words. 

As for the application of tools like Sora in movie production, Andy said that the peers he contacted still felt that they were neither particularly opposed nor particularly accepted.

According to Jiang Lai, there is no AI-generated image application in the movie industry at present. He told NBD that Tools like Sora are equivalent to pre-setting the post-production, so we don’t have to set up green cloth in the studio to shoot. In that case, everyone’s final presentation effect in the post-production stage is still in the imagination stage. After tools like Sora come out, they greatly facilitate the production of storyboards, and I can even complete the storyboards by myself.

“Therefore, most of us seem Sora as a tool, but we should actively embrace it. These tools will definitely have a very positive effect, and they will definitely greatly improve the efficiency of movie production, and solve the pain point of the mismatch between the pre- and post-production stages of the whole team.” Jiang Lai added to NBD.

Industry: “AI is hard to achieve complex emotional expression”

With the advent of AI text-to-video tools like Sora, it is possible that AI tools could eventually replace actors in some films. However, there are a number of challenges that need to be overcome before this can happen.

One challenge is that AI-generated videos can often look fake, especially when they are used to depict human emotions. This is because AI does not yet have the ability to fully understand and replicate the complex emotions that humans experience. As a result, AI-generated videos can often come across as being stilted and unnatural,Jiang Lai told NBD.

Another challenge is that AI tools lack the same level of creativity and spontaneity as human actors. Human actors are able to improvise and adapt to changes in the script, while AI tools are limited to what they have been programmed to do. This can make AI-generated films seem less engaging and realistic, added Jiang.

Finally, there are a number of ethical concerns that need to be addressed before AI tools can be widely used. For example, it is important to ensure that AI tools are not used to spread misinformation or propaganda. Additionally, it is important to consider the impact that AI tools could have on the livelihoods of human actors.

Overall, it is clear that AI has the potential to revolutionize the film industry. However, there are a number of challenges that need to be overcome before AI tools can replace actors.