I made a Music Video in Stable Diffusion AI with the Deforum SD Google Colab. I used Video Input with Video from Pexels, that I cut in Davinci Resolve to fit the music first and then rendered it Scene by Scene in SD Deforum Google Colab. I use the same prompt for most of the Video. Just in one Case I described the Scene to get a better Result. In Davinci Resolve i also added some zoom and movement to most shots to make the a bit more animated. The fading to Black and White and fading out effect is also done in Davinci Resolve and well as the rotation of the video in the second but last scene.
Music: Someone I Used to Know – Loving Caliber from Epidemic Sound
Join this channel to get access to perks:
https://www.youtube.com/channel/UCCKx8mAHiFus-XYQLy_WnaA/join
My Facebook group: https://www.facebook.com/groups/oliviotutorials.superfan/
My Newsletter: https://oliviotutorials.podia.com/newsletter
My Affinity Photo Creative Packs: https://gumroad.com/sarikasat
My Patreon Page: https://www.patreon.com/sarikas
All my Social Media Accounts: https://linktr.ee/oliviotutorials
Not a fan of trhe music but the visual artistry and concept is out of this world! Fantastic work Olivio!
WOW the potential of this technology is huge. Great work! š
this moved me to tears. simply sublime
Wow, really looking forward to the tutorial on this one!
0:15 I know where this come from… A stock footage of course! Here it is:
https://youtu.be/F61pyO80CCs?t=107
Beautiful. Haunting. Beautifully haunting! Olivio, you're the master! Can't wait to see your "How I Made This" video! āx10
video work feels a.i. historic first great work…
I've gotten into this AI Art passion of mine 3 months ago and just when I started to experiment with AI videos I see this masterpiece, this is just brilliant. the amount passion and pure experimenting that must have gone into this must have been astronomical. keep doing what your doing, your work is amazing. also I've learnt a lot from your videos already so thanks a bunch.
Noooooooo š¤Æ
You artist!!! Congrats Olivio for this amazing great job.
Thanks for sharing ā¤ļø
can u share tutorial please
Stunning, well done !
Good job.
Amazing video! This IS art, no doubt about it! Congratulations, very nice work!
Incredible! Amazing work!
Love to hear the story be hide the videošÆ
Whoa!
Do you have the Pexels clips assembled for comparison? Would be neat to see the before/after comparison!!
Create a video tutorial on how to do it
Looking forward to see short films made entirely with Stable Diffusion coming out the coming weeks ^^ this strange kind of transition might only be a temporarily artifact with those early AIs, but at the same time thatās also a very cool art style by the way
Awesome šš¼
Did you input the lyrics as prompts one by one and then do the interpolation? Or did stable diffusion make this video automatically?
I liked it a lot but I think it was not very creative because. In uncertain way it can be compared to someone applying filter to a stock video. It was a very beautiful AI art above those pre recorded video but it is only that
Great video. It's the idea that counts. As someone who's worked in film for decades, this is revolutionary tech. I've been coding my own effects in the past, but it was all non-AI. Shader type stuff. I only dove into stable diffusion recently, and loving it. I've had so many scripts in the past, that I'd reject due to being too large in scope. Unfilmable on a short budget. Now I can use a handful of AI's to produce absolutely everything for me. One writes a script. Another one makes an animated storyboard, the 3rd does a synthesized voiceover in any accent. And it all looks fabulous even without the post treatment. I'm still trying to figure out what to do with things like the depth pass. How to tap into the OFlow they're using to mess with motion vectors a bit. I also see that nobody has written a camera animation tool. It's all done semi-manually with spline to keyframe converters. Coding a camera exporter for Maya should be the easiest to do. Then it would be possible to have a 3D video relay some of the camera information back to DS, allowing for higher coherence. Only been coding for a week, and I'm already overwhelmed with ideas. And I love watching videos like this, to know what the others are coming up with. 2 years ago to produce something like this would've taken a team of people months of work. Now stuff like this can be produced in 24 hours. Soon, the prompt response will be so quick, that they'll hook it up to voice recognition blocks and have live AI driven shows. Interactive randomly generated stories with illustrations. AI Dungeon already does that to an extent, but it's basic. Just imagine what tools we'll have in a year! I haven't had this much fun since DeepFaceLab came out.
Excellent work!
AI video is born. Truly impressive.
Thatās freaking gorgeous.
Could you do a break down of your process, Iām sure weād all love to see how you did this!
Wow brilliant!
So moving and inspiring thanks for this š I love ai art and have been messing with it for months can't wait to try this myself thanks for posting a guide and on fb š
You never cease to amaze me with your videos, Olivio. This highlights a great feature of AI which is to make art videos and it does it extremely well!
Nice work!
Holy shit I'm sure it was a lot of work but this was an incredible production. Great scene selection.
I'm currently running Stable Diffusion 1.4 on my PC (using the WebUI) and still my brain can't understand how you managed to do this xD Amazing job!!
Looks great!š¬ Let's do it!
When you think those people don't exist!
I DON'T WANT TO LIVE IN I-ROBOOOT!!! š¤Ŗš¤Ŗš¤Ŗš¤Ŗ
Seriously…
Will we be allowed to sue if one day we see our own face on a screen saying or doing something we disagree with ??
Freeakyyyy …
š³š³š¤Øš¤Øš§š¤¬š§š¶š¶š
Waiting for the making of š
Will you do the "bĆŖtisier" like Toystories šš¦ š¤ and Shreck š¤¢š§ ?
Nice one!
How you use a video as an input in Stable Diffusion?
This is amazing!!!
š This deserves a making of.
Where I can learn more about stable diffusion?
You really should have separate channel for AI stuff. It was very cool to see Affinity tips and tricks but personally I cant stand more AI blurry nonsense. Glad you found new fascination – much luck. Sadly its time to unsub for me. I am not outraged, I am not writing this to make you mad. I know one person means nothing in the grand scheme of things. Just letting you that maybe separate channel would be good. I would gladly watch your Affinity videos when the AI rage passes.
This is beautiful.
How many hours of work?
What was your inspiration?
Both a technical and artistic achievement! Well done, sir š
Excellent, and amazing. Thank you for sharing your talent.
Great job, i'm assuming you used stock footage for the init video?
This was very impressive.
Wow Amazing!!
This is crazy impressive. š
Awesome !
Stunning. Will you be posting a tutorial about how you did that?
Amazing work!
Excelent work Olivio!
How did you get the consistency in shots?
Looks like this is from a memory of a person trying to think the scene from a long time ago.
Nice Job! "I use the same prompt for most of the Video" – what prompts have you used exactly?
beautiful, and it looks like an obscene amount of work went into it.
Fantastic! Congrats!
Is this a video input or fully generated? Crazy either way
One of the most underrated AI works out there right now. This work needs to go viral.
Absolutely amazing
Its beautiful literally made me tear up the music and art where great together
How?
Wait what!! š®
This is amazing!
I understand how to make still images but how did you make whole scenes?
Super impressed!!
Incredible! So I should use an app called stable diffusion ai to do this with my ai images?
Reminds me of Ahaā¦ take on me šš»
How long did this take
can you please make a tutorial on how to make one? I am a writer and I would love to make some of my creations come to life in a form of a video like this, please! are there any videos you recommend watching to learn this?
Son videos modificados con IA ..no son creaciones de la IA
Nice job being the first lid on the block t do this!
It's pretty low hanging fruit though
Foe each shot there's a prompt – 'barelegged girl shown from the waist down in a yellow summer frock walking away from the camera in a field of yellow flowers' or 'close-up of Caucasian hands flipping through square color photos' and a base image
Then you run the software and generate 200 iterations
Toss the bizarros and flip-book the rest and you have one shot
20 or 30 more and you have a video!
There will be a tsunami of this sort of quick 'n dirty work but in a month or so the really good stuff will come and it will dazzle!
What strength setting are you using?
Beyond inspiring. This is stunning.
Thanks for sharing. Here is our first teaser for a music video. We used the Warp Fusion notebook from Sxela.
https://youtu.be/i08x6InvJ38
Get ready to get your salary in visual mediums cut in half or even more.
Welcome your glorious AI future.
Wunderschƶn!
WOW! What a beautiful music video !š„° Awe inspiring !
Hello man, any tutorial for this? i tried img2img batching but it gives me different outputs for each image with the same settings
Amazing work man!! Deffo need tuts for this.
Your description is cool but , I Would love to see a tutorial for a video like this. Also an excuse to see another one of your creations.
very nice work here!
A – MA – ZING! Definitely I will try to do something like this! Thank you for inspire!
Incredible. There are dabblers and there are masters. You are the latter for sure.
disturbing.
All I can say is …HOT!!!!!!!! Fire!!!!!
I don't think you can do that with stable diffusion, unless you make all the images by hand and then tell it to improve them, otherwise the generation would not have continuity and the same textures and shapes in the images
Very nicely done. Bravo.
cool jobĀ”Ā” it is great
by any chance could you share the propt you used to achieve this type of look. Whenever I generate something the results are always horrific
oooawsome
make a tutorial on this please
Dude, itās a such good thing! How much time did you spend on this?
scramble suit
As a fellow AI art enthusiast I am amazed of what youve done!
But Im sure that from a mainstream point of view it still looks a bit like the character are cursed in the video haha.
That a first step after only such a short amount of time this tech is out, I cant imagine what we are going to do in a year or more.
can you send the settings file
Awesome work!!! it was made in 1080p from Stable Diffusion? or may have used any upscaler after? Thanks for sharing!!!
Whoa can we just appreciate how good this is and also can we please get a tutorial on how you did that cause this is sick
I love this it reminds of on Futurama when they had those instruments that projected the image. This is beautiful
awesome!!!!!! congrats!
please add the settings file so that we can follow along….Thanks for the help
This deserves a billion views. Exceptional.
What prompt did you use?? It looks amazing!
This was made with Pexels, not with Stable Diffusion. Very misleading and the result itself is meh
This should have wayyyyyyy more views than it does currently, that makes me sad about humanity
It looks good. I just wish you used AI for the music too. This song is awful. š