LogIn
Riffusion shows off impressive tunes.
In this boom of generative AI, we're used to it creating all kinds of stuff: code, images, 3D models, books, and more. Now, if you're bored of your favorite playlist and famous singers, you can get a totally new song with Riffusion.
What makes it interesting is that this Stable Diffusion graduate can not only generate a melody but also write lyrics and even sing – all you need to do is give it a prompt describing what you want to hear.
To achieve the result, Riffusion's creators Seth Forsgren and Hayk Martiros fine-tuned Stable Diffusion on spectrograms – visual representations of audio that show the amplitude of different frequencies. AI can go through the spectrograms and turn them into music.
“Users simply describe the lyrics and a musical style, and our model generates riffs complete with singing and custom artwork in a few seconds. From inspiring musicians, to wishing your mom ‘good morning!,’ riffs are a new form of expression and communication that dramatically reduce the barrier to music creation,” Forsgren told TechCrunch.
Now, Riffusion produces short clips only, and to my ear, the AI voice always sounds "split", but it is still pretty good and convincing, so give it a try here.
The last thing I've heard about the tool is that Forsgren and Martiros decided to commercialize Riffusion and closed a $4 million funding round led by Greycroft with participation from South Park Commons and Sky9. The improved version, with lyrics, is available for free.
If you want to see something similar but without lyrics, check out Google's MusicLM as well. It's an "experimental AI tool that can turn your text descriptions into music."
Don't forget to join our 80 Level Talent platform and our Telegram channel, follow us on Instagram, Twitter, and LinkedIn, where we share breakdowns, the latest news, awesome artworks, and more.
Keep reading
You may find these articles interesting