No threat to animators from these bizarre videos shared on social media today - very artificial and you would probably feel nauseous if you watch more than a few seconds of them
commentary on these videos may help you:
Video: No threat to animators from these bizarre videos shared on social media
These videos are very very bizarre if you pay a bit of attention to them instead of just going “wow - like - share”.
Video: . YouTube
ake this example tweet:
https://twitter.com/aaronmac64/status/1758216278962057510?s=19
SO MANY BIZARRE ISSUES IN THIS ONE SHORT CLIP
Look more closely. The way that couple is walking isn't natural. Or the way they turn their heads as they walk.
This one only seems somewhat natural because they are seen from behind and at a distance.
They are walking as if on stilts. There seems to be something wrong also with the legs of the man on the left. It looks like he is limping on his left side but somehow in a way that doesn’t interfere with the speed of his walking.
Like imagine you were walking on very short stilts just a couple of inches long. Not high heels., Toes not touching the ground.
There are other issues too - at the end of that short scene the couple in front of them dive under a kind of covering of a place bizarrely situated into the middle of their walkway - with another parallel walk to the right which suddenly stops, with the two divided off from each other with a pointless very low barrier.
What is this couple going to do 5 seconds after the end of the video? Dive under the roof in front or step over the barrier? What are the people going to do that are marching towards them on the right hand side there? Step over that small pointless barrier?
It makes absolutely no sense by way of city planning. And no sense in terms of any kind of a plot or direction.
Also at the start of the clip the red cover to the right of the girl is at about the level of her right hand. By the end it is above her head.
Also it’s not clear how she manages to get past the red cover as she seems to be heading straight for it. She also seems to be heading straight through the snow in sequence leading up to this image.
Those branches on the left on the trees are really weird too.
By now the red cover is above her head or a similar height. It is not really clear how she fits in geometrically with that cover, it’s a bit ambiguous like a copy / paste from another scene into the image:
Now it is well above her head:
And this shows the strange way the walkway they are on suddenly stops with a low barrier which they presumably have to step over to a parallel walkway with people on it who also seem to have encountered a bit of a strange end to their walkway too.
It’s like an Escher print.
NO WAY TO GET THE AI TO FIX THESE ISSUES
You can’t fix it either, there is no way to say to the AI “That eatery in the middle of the walkway makes no sense please remove it”. It doesn’t know what anything is.
These bizarre videos only seem a bit plausible because they are new. Once people have got over the initial "wow" I expect they will start to seem as bizarre to them as they look to animators right away.
I am not an animator but I have done programming using OpenGL for music software as well as software to make virtual flowers and trees so I notice things like lighting, shade, and movements.
It immediately just looked very bizarre. I could see much of this right away on the first watch.
Like that man's stiff left leg which didn't move properly yet didn't hinder his movement. Kind of limping but without slowing down, very bizarre. I think that was one of the first things I spotted and their weird jerky movements and the floating stilt like appearance
It's like it's all made of helium balloons with no weight to it. In the next video we look at the monster's head and tail are just floating, not resting on anything or supported by anything.
If a CEO thought these were good animations he or she should get another job and won't last long at the head of an animation company.
Would steer it to a flop. The movie or animation would get rotten tomatoes reviews from anyone who has ever done any animation. And probably half the audience would complain of nausea after watching it.
GENERAL ISSUE WITH ALL THE ANIMATIONS - CHARACTERS LOOK TO ANIMATORS LIKE THEY ARE FLOATING - LIKE HELIUM BALLOONS LIGHTLY TETHERED TO THE GROUND
Then more generally all of the videos with characters in them look like the characters are floating, they have no weight, they are like helium balloons that are tied to the ground.
This is especially obvious with this one of mammoths.
These mammoths look like they are floating and they don't leave any footprints in the snow OpenAI (@OpenAI) on X
https://twitter.com/OpenAI/status/1758192960116785459
It’s not just me, animators are commenting saying the same thing.
QUOTE This looks like animation that someone who is just learning does. It’s floaty and has no sense of weight, timing, interesting spacing. Literally lacks all the principles of animation. Guess it’s fine for garbage content.
https://twitter.com/mattsackley/status/1758302893738783189
It is most obvious in the mammoths perhaps because mammoths should be heavy.
They looked like mammoth sized helium balloons lightly weighted so they could touch the ground.
ALL of them are like that. Floating, no sense of weight.
ANOTHER VERY BIZARRE SCENE WITH A MONSTER WITH HAIR GROWING FROM ITS HORNS, AND A CANDLE FLAME THAT DOESN’T FLICKER, LOOKS LIKE A BUNSEN BURNER IN A DRAFT FREE CABINET AND NO CHANGE IN THE LIGHT THROUGH THE CANDLE AS IT QUICKLY MELTS
And this one
QUOTE it has fur growing out of its horns. and it grows an extra finger as it reaches for the candle. you people are so blind lmao.
https://twitter.com/AporicWares/status/1758539010572505190
That creature
grows an extra kind of finger in the middle of its palm as it reaches out to the candle.
candle flame doesn't flicker.
has fur on its horns.
lighting on the candle illuminating it from within doesn't change as it melts.
candle melts far too fast in a way that doesn’t make physical sense
super artificial and very bizarre.
I think probably people who have no idea about animation just look at it for a few seconds and go "wow" because they haven't seen anything like it before.
But they would feel sick if they watched a longer animation of a few minutes of this stuff, it is very weird and the cumulative effect would be difficult to take.
FEEDBACK FROM ANIMATORS COMMENTING ON TWITTER
Also - you can’t direct the animations. It is just whatever it produces.
QUOTE
Feedback:
-go back to storyboard to define the point of this scene, why the character looks down.
-don't forget to add blinks
-remove the frown
-make the side horns the same material as the top horns
Oh right, you can't react to feedback w AI. Bin it. Hire an art team next time
https://twitter.com/ChrisReeves_3D/status/1758475265594859854
For the ones that are suppose to resemble movies with real people, you couldn't say "cut" and then ask the actor to move in a different way.
For the ones that are supposed to resemble animations, you can’t do anything if you want to make slight changes. Just try a different prompt in words and keep trying in the hope eventually you get something remotely resembling what you wanted to create.
More reactions:
QUOTE This looks like trash.
https://twitter.com/ScratchyDerose/status/1758294672655388918
Even if I didn’t know this was AI. It’s still super ugly and weird imo
https://twitter.com/villainousbolt/status/1758534016163348575
As someone who used to do animation there is so many things wrong with this scene.
https://twitter.com/CarolineDamian/status/1758391552861442133
SNOW PUPPIES ANIMATION
This one one may seem to work because it is really short and perhaps because they are so cute you don't notice the glitches.
This graphic shows one of the glitches. The puppy on the left looks like it has broken its front right leg.
Screenshot at this point OpenAI Sora - snow dogs
Generally the puppies don't quite seem to fit together in 3D space. They would overlap in 3D at times.
.
As with all the Sora animations the puppies don't look as if they are connected to the ground there's a disconcerting floating quality to them. Again them being so cute may mask that, also the clips being so short. In a long animation you'd probably find it disconcerting.
And there is no way to direct any of this. E.g. you can't say "Fix that puppy's broken leg". You can't even say "please do an animation of puppies playing in the snow where none of them have broken legs" because Sora clearly is not able to figure out how puppy legs are supposed to move.
It's not unlike the bizarre limp of the Japanese man in the first video where he has a very stiff left leg and moves it as if limping except that he is not slowed down at all, doesn't favour the right leg and walks as brisky as if he doesn't have a stiff non bending left leg.
The animations look okay to people who don't look closely and just go "oh cute" or "wow" but they would make any movie company that uses them the laughing stock of the animation industry.
It is possible that they fix these glitches, I don't know. They may be more serious to fix than it seems since there is nothing there that knows what a puppy is or snow or legs.
If it is possible to create virtual realistic animated scenes that would be convincing to animators in the future then they might use them sometimes as with the virtual orchestras.
But not replace the animations by animators or the movies of real actors. The issue is that there is nothing there that knows anything / understands anything / can put in any realistic interpretation.
THERE IS SOME USE FOR AI ALREADY - DEEP-FAKE LIKE METHODS TO CHANGE DIALOGUE FOR A SCENE WITH A NEW VOICEOVER FOR JUST ONE ACTOR’S PART IN A SCENE - BUT NO WAY THAT THIS IS ANYWHERE NEAR SUITABLE FOR ENTIRE VIRTUAL ACTORS
They can use deep fake-like techniques to change the dialogue of an actor in a scene already shot in a passably presentable way.
I don't think they are yet at the point where they can do virtual actors that would fool someone expecting real people in the movie except at a distance e.g. maybe a crowd scene with the people very small.
They can do still images now that often could stand in for a photograph of someone, for most viewers anyway, but once they move that's when they reveal their artificiality. Of the ones I've seen.
IF WE DO EVER GET CONVINCING ARTIFICIALLY GENERATED ACTORS OR CHARACTERS - LIKELY TO BE OF LIMITED USE MAINLY BY LOW BUDGET MOVIES AND GAMES - ANALOGY OF VIRTUAL ORCHESTRAS - HAD REASONABLY CONVINCING VIRTUAL ORCHESTRAS FOR MANY YEARS NOW
But if they ever do achieve this - maybe they will in the next few years - we have had AI generated voices and especially instruments for many years but still use live musicians for most movie scores even. Sometimes mixed in with virtual instruments.
The music industry has had virtual orchestras for a decade or more now. They are now at the stage where for a non musician they can seem quite convincing. If the composer does a lot of work into adjusting the individual notes and parts in the virtual orchestra it may even seem convincing to a musician, sort of.
And there are still live orchestras.
I explain here about how we've had this for composers starting in the 1950s and nowadays automated composition can create music in many different styles
We have virtual orchestras you can use to turn your notes on a sheet of paper or nowadays on a score in a software program directly into a virtual orchestra complete with all the instruments in the orchestra playing away, trumpets, French horns, cymbals, flute, oboe, violin, double bass, piano - all there playing on your computer but not a human player in sight. Singers too, virtual choirs.
Yet we still have musicians, singers, orchestras.
This is an example of a modern video game that uses live orchestra, Assassin's creed
Video: Assassin's Creed Symphonic Adventure: World Premiere concert aftermovie
Most of the big video games do. Smaller companies can use virtual orchestras.
I don't think there are any examples yet of voice actors or actors that are synthesized. Except for some robotic voices.
. Synthetic Voice Actor - TV Tropes
Same also for singers, a composer doing a mockup might use a synthetic voice but not for a real performance for the public.i
It works better for choirs than for solo voice in singing too.
Same also for virtual orchestras, it's especially obvious that it's synthetic for soloists.
I can't see it working anyway. In film scores and movie scores there is nothing to stop a composer using virtual instruments throughout and occasionally they do. But it is rare.
I mean in high budget ones.
They might do it for a special effect, to get the synthetic sound.
GOOD SUMMARY BY AN EXPERT ON REDDIT OF HOW VIRTUAL INSTRUMENTS AND ORCHESTRAS ARE USED IN MOVIES AND VIDEO GAMES
This is on Reddit but it is a good well written description of how virtual instruments and virtual orchestras are used in movies and in video games. It was written 4 years ago but I don't think much has changed since then. Composers still use live musicians and often a hybrid of virtual and live - some virtual instruments are used because composers like the sound of them in particular contexts.
But video games and movies mean that musicians have plenty of work for them in the movie and game industry!
I expect the same to happen with animation. They may use AI for some particular effects.
I am not sure these animations would be used at all in any commercial movie they are just too stilted to be useful but maybe later they will.
I’d have thought the most they’d be used for would be for brainstorming.
To try out ideas. The AI would get it all wrong but some of its mistakes might suggest new ideas for animations or movies.
Or maybe to do it as a parody or pastiche of AI work.
PAPERCRAFT VIDEO IS PERHAPS THE BEST OF THESE - SO OBVIOUSLY ARTIFIICAL - BUT EVEN THEN I’D PROBABLY START FEELING NAUSEOUS AFTER A FEW MINUTES OF IT BECAUSE OF THE UNCANNY VALLEY EFFECT (THE AWKWARD CONFUSING GAP BETWEEN OBVIOUSLY ARTIFICIAL AND REALISTIC ENOUGH TO BE CONVINCING)
This one is perhaps one of the “best” just because it is so obviously artificial still I think I’d start feeling nauseous if I watched an entire video of say 10 minutes of this.
https://twitter.com/OpenAI/status/1758192962755051792
There is something very strange about the lighting. There is no ambient light. All the fish have exactly the same illumination, no shadows passing over them, no changes of light and shade which you’d have if they were really made of papercraft.
All the turtles are identical. Every segment of the sea horse has the exact same lighting and shading.
I am not an animator but as a software developer, I have used Open GL to make bouncing balls and other visuals for my virtual metronome and so perhaps that’s why I have a bit of an eye for this.
It is not ray traced and it isn’t any sensible cartoon style either. It is in that uncanny valley between real and stylised and to my eye this makes me feel nauseous to watch it.
It’s not just me, so many on Twitter are commenting saying how strange, ugly and artificial these videos are.
EXAMPLE OF GRAPHICS MADE WITH BING CO-PILOT - THEY CAN BE FUN FOR SOME THINGS BUT ARE OFTEN VERY BIZARRE
Of course some will say “wow” and they may use them for something.
I use Bing CoPilot for fun graphics for entertainment as with this one I did recently.
The background graphic here is made with copilot, with Trump / Putin / Zelensky pasted on top. Because of the topic “Cloud Cuckoo land” and there isn’t any preset idea of what Cloud Cuckoo Land would look like then it kind of worked I thought :)
The world "cloud cuckoo land" refers to a utopian land in the skies (a wordly paradise in the skies) and goes back to a mythical kingdom, "cloud cuckoo land" or Nephelokokkygia in created by birds in Aristophanes play "The Birds" in which the birds become gods with the help of humans. The Birds (play) - Wikipedia In modern times it goes back to Schopenhauer who used the word Wolkenkuckucksheim or Cloud Cuckoo Land inspired by the Greek idea.
Wikipedia summarizes it as:
QUOTE Cloud cuckoo land is a state of absurdly, over-optimistic fantasy or an unrealistically idealistic state of mind where everything appears to be perfect. Someone who is said to "live in cloud cuckoo land" is a person who thinks that things that are completely impossible might happen, rather than understanding how things really are.[1] It also hints that the person referred to is naive, unaware of realities or deranged in holding such an optimistic belief. Cloud cuckoo land - Wikipedia
I thought the bizarre graphics fitted rather well with the over the topness of “cloud cuckoo land”
There are so many strange things in that image. Those bizarre cloud coloured tree like structures holding up the cloud at top right. Strange curlicues that make no sense. Why does it have very huge mushrooms everywhere larger than trees (not biologically possible)? Multiple rainbows above the clouds (you only ever have one rainbow or sometimes concentric rainbows).
Sometimes it is okay for fun silly still images - nowhere near what a graphical artist would do, but if you want to just make an image for fun then I find this AI quite useful.
But not going to replace graphic artists for that either.
However to my eye these new animations are unusable. Perhaps people might use them in TikTok? If so it will be interesting to see if they work in that medium, perhaps for a while until viewers get used to them and then see them as artificial and no longer want to watch them.
It’s the same for all the AI animation I’ve seen, none of it is even remotely close to fit for purpose even for something fun to share in a blog post, well, not for me :).
Maybe some day it will be, then I expect it will be used sometimes similarly to the way virtual orchestras are used in movies and in video games, mixed in with real orchestras with some instruments played virtually because the composer liked the virtual sound for those particular instruments and uses them for effect.
This will require updating in light of Sora 2's release:
https://x.com/naegiko/status/1973171308868542578