5 questions about AI and music

Yes and no; it depends on your perspective. An AI system can generate musical ideas, parts of a song such as lyrics, a melody line, or a bass part. But it can't do this on its own without a human pressing the buttons. AI can only be self-learning when it is fed with data, and what goes in determines what comes out: feed it with jazz, and you probably won’t get death metal. 

One famous name in AI-generated music is the French scientist and composer François Pachet, who has been working in this area since the nineties, first in a lab for Sony and now for Spotify. His main achievement is the Flow Machine, a neural network that generates music based on musical datasets. discovering patterns in the music you feed in and using this to create something new. He explains how this works here.

AI can generate the pieces of a jigsaw puzzle that can be assembled into a song, but the assembly has to be done by a human. So far, no AI is so good that it can compose a complete song to rival works by Earth’s great human artists.

The teams in the AI Song Contest use different techniques to write their pieces. In this series, one of the teams reveals their ultimate goal to create a hit.

Well known musicians have flirted with AI in different ways. Ariana Grande has used MI.MU Gloves to manipulate the pitch of her voice during live performances, creating whole harmonies with the twist of a finger.

The British band Muse didn't make music with AI, but they did use it to create a live video clip. In the video for Dig Down, an algorithm searches the internet to find videos that match the lyrics of the song.

Several international musicians, including the Belgian artist Stromae, have made use of Pachet's Flow Machine. In 2018 this resulted in the very first AI album, Hello World.

The American musician Holly Herndon goes a lot further, and has been composing with computers since the early noughties.. In her most recent album, Proto, she took this to a whole new level, bringing her own 'AI baby' into the world by training it with her own voice. She wrote the music herself, sang it, and the AI recorded it, time and time again. After a while, the system generated new sounds and melodies based on all this input. She called it Spawn, and regards it as a new member of her ensemble, since it sings along even during live performances.

Holly Herndon’s work is best suited to the somewhat adventurous music lover, but musicians and technologists CJ Carr and Zack Zukowski take things even further. This Berlin-based American duo developed an AI program called Dadabots, fed with death metal, jazz, and punk. The algorithm pulls out the most common musical elements and sequences, and produces new death metal based on them. You could say they’re an acquired taste, as their entry for the AI Song Contest mixes metal with pop.

One of the real pioneers is David Cope, now 78, a composer, scientist, and former music professor at the University of California. He started experimenting with AI and music out of necessity in the eighties, when he was suffering from writer’s block. After years of monastic work he created new AI pieces in the style of Bach, Beethoven and Chopin that are almost indistinguishable from the real thing.

Artists like Kraftwerk, Giorgio Moroder, and Aphex Twin are pioneers of electronica, a collective name for music made with electronic instruments such as synthesizers or laptops and played by people. 

AI goes a lot further: it’s not just an instrument, but acts almost like another artist in its own right, generating its own music and providing compositions, melodies and lyrics using computer algorithms to learn from data and create its own autonomous input. Instead of the musician 'playing' the technology, he or she curates the AI output and decides whether or not to use it.

It’s good to see that some of those electronica pioneers are also flirting with AI. The British triphop band Massive Attack, for instance, has had its 90s classic Mezzanine remixed by a neural network.

AI is a tool and a huge inspiration generator. David Bowie was assisted by a computer when writing his lyrics in the nineties, and created his own AI of sorts. He developed Verbalizer, a program into which he typed text in columns and which then huddled the whole thing together and made new sentences with it.

What the computer does here is to surprise you with new suggestions – and as a musician, you then decide which ones to use. It creates lyrics, melodies, and harmonies, all the ingredients that make up a song, and gives you thousands of ideas in no time. You’ll never have to wait for divine inspiration again, and the tool is a great way to get your creative juices flowing.

AI also gives people who don't play an instrument the chance to express themselves musically. The AI Song Contest shows that many scientists and tech nerds have been able to create songs: without AI, they would never have dreamed of doing so.

It’s also worth making a distinction between music that serves an artistic purpose, and that which functions as wallpaper, for example in shops and videogames. AI Song Contest jury member Ed Newton-Rex says no one needs to be pressing the buttons for this kind of music. 

He founded the composition company Jukedeck ten years ago, to make AI music for video and other media. It was acquired by ByteDance in 2019, and has created more than a million original pieces of music. Newton-Rex says that AI can make soundscapes, but it can't create something with a purpose, because it has no idea what it’s doing. But artists do.

There is no unambiguous answer to this question. Some scientists are concerned about the erosion of cultural diversity in a music world that’s working more and more with algorithms. As this music becomes more popular, they fear that it could acquire a western-dominated flavour because it’s easy on the ear. AI Song Contest juror Vincent Koops acknowledges this danger, and says it’s  especially important if musicians all start working with the same datasets.

But if you start creating your own datasets, you can build in as much cultural diversity as you want. For example, Koops enjoyed seeing how a contest team from the Netherlands trained their AI with old Dutch folk songs. Everything is possible: populate your dataset with Hindi tunes or obscure squeaky music, and your AI is as diverse as the music you train it with. Even if you use an existing dataset, not all of these are based on western popular music. 'There are initiatives that propagate diversity in music,’ Koops says. ‘You often see them in university projects, more focused on an audience than on commercial interest. But they are there.'