Tuesday, Jun 21, 2022 • 33min
AI in music: here’s what you need to know
Play Episode
Artificial intelligence is quietly transforming the world as we know it - and AI in music is no exception. From song-identifying apps to composition tools, all the way to lyric generators and production platforms, AI is reshaping the way we create, share, and even discover music. In this episode of The Mix, Cari, Silvia, and Stella share their professional experience and research into the topic of AI in music, as well as chat with AI music expert Valerio Velardo about his thoughts on these exciting innovations and predictions for the evolution of the industry. Join us as we explore the world of AI in music to see how it is already being used and what’s in store for the future.
The Mix is a Musixmatch Pro podcast.
All music content for this episode was composed and produced by Pierfrancesco Melucci.
Read more
Talking about
Speakers
(4)
Valerio Velardo
Cari Quoyeser
Silvia Olivieri
Show more
Transcript
Verified
Cari Quoyeser
00:00
Hello and welcome. You're listening to The Mix a
Musixmatch
Pro podcast hosted by yours truly Cari Quoyeser
, an artist community, and services.Share
00:13
Silvia Olivieri in publishing.
Share
00:16
And Stella Tavella, in
AI.
Share
00:18
Here to bring you the scoop on what's new in music innovation to help you navigate the modern music industry. Enjoy.
Share
Silvia Olivieri
00:27
In today's episode we will talk about the application of
artificial intelligence
in music composition. AI
has become an increasingly popular mode of co-creation.Share
Stella Tavella
00:38
Cool, right?
Share
Silvia Olivieri
00:39
What does that actually mean?
Share
Stella Tavella
00:41
The three of us will break down the most common questions about
AI
in some writing questions like will AI
compositions replace human compositions altogether?Share
Silvia Olivieri
00:51
How do artists feel about
AI
and how are they currently using it?Share
Cari Quoyeser
00:55
What are the implications of using this kind of technology?
Share
Silvia Olivieri
00:58
And how do we even begin to establish copyright ownership?
Share
Stella Tavella
01:02
Sounds like a lot?
Share
Cari Quoyeser
01:03
Don't worry, there's a lot to entangle. But along with our guest
AI
expert Valerio Velardo, we're here to help guide you through every step, starting with the most basic question.Share
01:15
So what is
artificial intelligence
? AI?
Share
Silvia Olivieri
01:21
Let's start off easy and define what
AI
means. If you're not an expert or familiar with the eye, you might think about robots taking over the world. It's not really like that.Share
Stella Tavella
01:32
Artificial intelligence
is the simulation of human intelligence processes by computer systems. It refers to the ability of the machine to learn and keep improving its performance without the necessity of humans explicitly programming it for a particular outcome.Share
01:53
Artificial intelligence
comprises a set of different techniques.Share
01:59
A subset of
AI
is the field of machine learning by which computers make use of statistics and logic for the learning process.Share
02:08
Another subset of
AI
and machine learning is deep learning where with the use of a large amount of data and sophisticated algorithms, the machine is able to autonomously train itself without the need of having specific engineered features as input.Share
Silvia Olivieri
02:26
So
AI
is different applications in music. It helps enhancing consumer experience, but also it gives machines the ability to become music co creator. Today we want to discuss the ladder.Share
02:38
While writing music is an expression of human creativity, music as deeply mathematical composition. The rhythm scales, intervals and harmonies all are based on mathematical structures.
Share
02:57
First studies on
AI
and music date back to 1980 but the recent developments have been possible thanks to the advances in both technology and computation in conjunction to this, the AI
. Now as an easy access to later dataset and sound sources that he can study and from which you can create new music.Share
03:18
This is called
data mining
and it's the extraction of contents from a database. In simple terms where the AI
is taking the data to study from. Now, we could open our own new chapter on this and trying to analyze what happens if AI
uses copyrighted data to develop. However, today's episode is not about that as we want to concentrate on our AI
is every artist right now.Share
Cari Quoyeser
03:43
So as our tech and
AI
expert Stella, you would probably be the best to explain how this is happening.Share
Stella Tavella
03:48
Well actually there is some really interesting software on the market. Currently some of them, for instance, can translate vocals into any instrument. So maybe if someone as tons of ideas but not the classical training or time to write out the whole score, they'd still be able to capture and express them.
Share
Cari Quoyeser
04:09
Yeah, I actually saw an advertisement for a phone app that you can the tune and it generates the music, which is so crazy.
Share
Stella Tavella
04:16
Additionally, there is other software that maybe don't total generate the whole tune but that are able to play piano or assist in the creative process in other ways.
Share
Cari Quoyeser
04:27
Yeah, it's wild to think about the possibilities that are arising with this new technology.
Share
04:31
But I also think it's really important to recognize for song writers or producers or anyone in that sector that most of us are using
AI
Softwares already right now and have been for a long time without realizing that it was... AI!Share
04:45
So in the studio, things like auto tune, auto leveling, auto mixing and mastering or any instance really where the computer learns were a note picture level should be relative to the others and implements that learning autonomously is
AI
.Share
04:59
Applying this general definition. There are a ton of other super practical uses of
AI
I for instance, with Stella's help of course, have used an AI
software that intelligently separates individual instrument tracks from a master copy of one of my songs.Share
05:16
This is really useful if perhaps you've lost access to the stems and still want to edit and remix your own track. Far from robbing me of my own creative process, softwares like this, save hours of time, they make our jobs easier.
Share
05:40
I think we can argue that
AI
is serving a number of different types of needs. A great example of this however, is Amper Music and Dudek, who are created with the purpose of offering low-budget compositions for background music and vlogs and videos help people that perhaps can't afford music licenses of manmade compositions.Share
05:59
Additionally, to pull it back to creatives,
AI
compositions can serve as inspiration for songwriters and through human collaboration can be crafted into songs indistinguishable from the ones created through the typical process.Share
Silvia Olivieri
06:14
How do we feel about
AI
being used to write music?Share
Cari Quoyeser
06:22
As we mentioned before, there's a big difference between using
AI
to level your music or quickly tweak a sharp note than it is to use it to generate new ideas or write full compositions entirely for that matter. For the latter application, AI
can be kind of a controversial subject.Share
Stella Tavella
06:39
In fact, even
Cari
and I have different perspectives on this matter as two very different types of creators. Cari
you're more of a, shall we say, analog musician and I'm a producer and computer scientist that makes electronic experimental music right?Share
Cari Quoyeser
06:59
Absolutely. I definitely see from my end of things a pushback for musicians that make music in the more traditional way like myself, but I think comfort levels vary. For instance, personally I don't mind
AI
creating harmonies, riffs or accompaniment, but I get a little bit weird about AI
writing lyrics as lyrics are really important to me, specifically as a songwriter.Share
07:22
All this being said, I know some artists like you, Stella, that use
AI
to create your art and as you've explained it to me, AI
is something that you collaborate with rather than something that writes for you. That's not to say however, that there is an AI
software out there doing just that and doing it successfully.Share
Stella Tavella
07:39
So from my perspective in the world of experimental music and production, I can see that there is a lot of excitement about
AI
being used for music creation. There have been a lot of well-known artists that have been making use of this technology for quite a while now. Artists such as Holly Herndon
or Ash Koosha
, for example.Share
08:00
Since this is a very experimental field of music, I can also see that producers who aren't really familiar with these tools yet are curious to learn more and excited to try this type of software for themselves.
Share
Silvia Olivieri
08:15
With technical musical skills become obsolete, to what degree?
Share
Cari Quoyeser
08:23
It's hard to say to what extent
AI
will progress. Currently, it's used widely in making our jobs of creating and producing easier. But I don't think AI
could ever replace technical jobs altogether. For example, I know very well how to record mix and master myself but I'll still pay for an engineer, producer to help me do it.Share
08:45
Having that extra body in the room. They're trained ear their impression and their emotional reaction to a take is invaluable to making sure the track is the best it can be a I can tell you what's technically correct but sometimes having a trained person there can pull the best performances out of you. At least that's been my experience.
Share
Stella Tavella
09:06
So like every technology that has ever existed. The initial concern is that humanity will stop learning and become too limited by tech. I say this is like when humans invented writing, they needed it in order to avoid remembering everything in their minds.
Share
09:24
Writing is a powerful tool that allowed all humanity to be informed and educated faster. It did not make people less smart just because they could read things instead of remembering them.
Share
Cari Quoyeser
09:36
When
AI
helps you write the song, who gets the publishing rights?Share
Silvia Olivieri
09:48
So before starting to navigate such a complex topic, it's necessary to understand better what the scope of copyright production is. We'll try to make it simple, I promise.
Share
09:60
First as the work is considered to be the result of the others intent and labor, copyright ensures the protection of the work by granting the author both moral and economic rights, moral right is the right to be considered as the author of the work as well as the right to object to any alteration of such work.
Share
10:21
The economic rights instead ensures exclusive rights, such as copying, distributing and licensing the work, getting the money, let's say. Because copyright protects both moral and economic rights, the inclusion of
AI
music leads to two main problems. The first one is ethical. Can the AI
entity be considered an author and consequently be ensured more protection?Share
10:48
The second is economical. A machine doesn't have any economic interest. Thus, copyright wouldn't be able to incentivize the creation and development of
AI
music. And how do we pay it? If we assign the rights to a machine, then the machine needs to be considered a fully fledged juridical entity and be able to take decisions and responsibility for the work.Share
11:11
Does this mean we need to consider
AI
the same level as humans? This has a huge ethical implication to now, we don't want to make things more complicated than they already are, and we want to use this space to educate and push whoever is listening to research and be more at ease with new technologies, we don't want to scare anyone here.Share
Cari Quoyeser
11:33
In keeping all of this in mind, what's the answer to the question?
Share
Silvia Olivieri
11:41
At the current state of the technology, considering the possibility of a machine-author seems to be possible only on a hypothetical level, granting no protection at all is also impossible. Why? Because denying copyright protection would mean denying
AI
the protection it needs to keep improving.Share
11:59
Indeed, without any protection there would be no incentive for
AI
music to be further developed. A lot of people might think, "Ah well, we have so many musicians, we don't need AI". That's true, but it's also true that you can't stop technological developments, so it's better if we learn to live with it and exploit it in the best way possible.Share
12:20
Also, given what we have discussed so far, we are not yet at the level where the
AI
machine is doing everything. Instead, we are in an exciting moment where artists use AI
as a collaborative tool, inventing new sounds and challenging their creativity. So what has been recently discussed in both EU and US legislation is that AI
is a tool, like a camera for a photographer. You wouldn't run copyright protection to the camera.Share
Cari Quoyeser
13:01
In the next and final chapter, our
AI
manager Stella and I were lucky enough to sit down with AI music expert: Valerio Velardo. Besides having 10 years of experience in the field of AI
as an engineering consultant, Valerio is a classically trained musician with a PhD In AI
Music. Valerio founded Mela drive a company that creates AI music for video games.Share
13:26
He has two prominent podcasts, The Sound Of
AI
that educates his listeners on the hard and soft skills necessary to create AI music for themselves, and The Sound Of AI
Tech And Society, that explores the impact of AI
on society. If that's not enough. Valerio also founded and currently runs the largest AI
audio and music community in the world.Share
13:48
Here he is: Valerio Velardo.
Share
14:01
Hi Valerio, thank you so much again for being here. We're just gonna jump right in if that's okay with you. And the first question we have is, in your opinion, what is the potential of
AI
to be used in composition? And how will this help musicians and songwriters in their creative process?Share
Valerio Velardo
14:20
Yeah, I think there is some big potential there for a I music competition and up until now we've only scratched the surface I believe. And that's because we are mainly concentrating on, in a sense, trying to replace musicians, trying to create some applications that can generate music and, in a sense, try to replace what composers can do right now.
Share
14:47
But I think the real hidden gem there is the possibility of expanding what's possible for musicians, namely the possibility of creating a sort of like infinite version of the very same piece of music.
Share
15:03
That's because potentially you could use an
AI
creative music system to pass it a sort of music scripts that then can be implemented in a number of different actual implementations, musical implementations that potentially respond to different inputs, external variables, like for example, the weather, your emotions or some sort of like emotional setting, for example, in a video game. So I see that as the most interesting application of AI
music generation in the future.Share
Cari Quoyeser
15:40
I can see why! Personally my the gears are turning in my head when I start thinking about all the different ways we could create art with these kind of variables. But to change gears a bit, I was wondering in your opinion, why do you think
AI
is more commonly used in certain genres over others? For instance, Electronica or... something like that?Share
Valerio Velardo
16:03
Well, I think right now, the the way
AI
is used in music is mainly for a music production, is mainly for music production, as said. Just like for example, for automatic mastering, we have like a few companies that do that and do that like reasonably well, like for example, Landr
. Right?Share
16:23
And I wouldn't necessarily say that it's mainly used for certain types of music music genres, but in a sense, some of this like a land themselves better to like
AI
. And that's because for example, for a synthesizer, synthesizer, synthesizer-based music, so it's easier to have an AI
that can help you in a number of different things.Share
16:49
Like for example, the production part of the composition, if you will, as well as the possibility of generating new sounds that are completely unheard of. Right?
Share
17:00
And that's a little bit more difficult if you're talking about some music that's a little bit more traditional, like for example, if there's some sort of art music, contemporary music. Because you don't need that type of extra help there where like
AI
really shines like for example generating new sounds because you're using acoustic instruments.Share
17:19
But nonetheless, I believe like that there are many possibilities in that aspect as well. Things like we have to like make a distinction here between using
AI
for purely compositional purposes, in other words, in order to create music or help you create some music to help you create the score itself.Share
17:40
And on the other hand, adding
AI
helping you with the production part of things like for example, mastering, mixing or creating new sounds and these sort of things. And for the latter situation, I believe that electronic music is... can have more applications.Share
Cari Quoyeser
18:02
How do you expect - and you kind of already answered this a little bit - but how do you expect
AI
to be positioned in the music industry in the future as a product?Share
Valerio Velardo
18:11
Yeah. I think right now, the biggest obstacle that we have in
AI-generated
music is the fact that we don't have a market yet for that. And there are many reasons for that. Number one probably being that the technology still, like not up to that level, it's not as good as a human being, would actually compose some music, right?Share
18:36
But there's also another aspect which is a lot of inertia from users and especially like bigger companies, I used to work like in generative music for video game music and it is really difficult to go to video gaming companies and tell them, hey, "Let's just like switch the way that you create music from." Your traditional way to like this completely crazy and completely like new way of doing music helped through
artificial intelligence
.Share
19:06
So I believe that the main problem like moving there would be to have people, the stakeholders actually using this technology and for that to happen, we have to create products which makes sense and are not just some really cool uh like technologies out there.
Share
19:27
And in that respect, I believe that probably the most interesting work that will open the way for your music generation is the work that people are doing within social media for example, where like there will be the possibility for normal users like
TikTok
users or Instagram
users to add some sort of customized music, very simple music, but quite effective and very personal, which I think is the main thing.Share
19:55
So having the illusion of having something that's completely geared for you and that you had a say in terms of like the compositional process. Right? And for me that would be probably the most interesting app near-term application.
Share
20:13
So you have like some big social network company that actually adopts music generation
AI
for enabling their users to create music that will the then they could actually couple with the content that they create on that platform. I think this is an extremely interesting aspect and I see a real market potential for this, but then there are also other applications, but this would be for more expert musicians.Share
20:47
And there I see the possibility of some sort of like enhanced doors for example, or enhanced music scoring systems which will help you for example create melodies that could potentially understand like the way you create music and streamline your music creation process by offering you like new melodies that are in your style or perhaps different from your style and you can just like pick up those ideas and use like what works best.
Share
21:19
And so in that way you could have a sort of like sparring partner where like
AI
is just like helping you providing you like with like snippets of music that you can actually use. So this is also like something really interesting.Share
21:32
And then there's the third aspect that I believe it's this the one that I'm the most fascinated by, and that's this idea of like taking like my music, my musical scripts and then extending them. Within for example, an interactive piece of media, like a video game or a virtual reality application, where I would create a musical script and then the AI could extend that and implement that into different emotional settings depending on what the end user is actually experiencing. And that for me is the most interesting part. But I think we are.
Share
22:13
It's gonna happen probably, but not right now.
Share
Cari Quoyeser
22:18
Yeah. Well, you know, I can see I can certainly see that interest towards
AI
and AI-technology
is growing, and as long as those communities are strengthening and growing in numbers that it's going to be something we're going to see more of in the future anyway.Share
Valerio Velardo
22:31
Yep.
Share
Cari Quoyeser
22:33
There seems to be a a general feeling in the artist community, in the songwriting community that
AI
is going to replace the need for skill, the development of skill. And I'm trying to change my own mind about this personally because I have some aversion to AI
writing lyrics personally.Share
22:57
That being said, I have to acknowledge the fact that there's a whole world of opportunity that's open to me. Like for example, I studied modes very briefly in university, but it's not something I retained necessarily. And having a technology that could take my song and show me what it could be, really widens the variety of music I'm able to create.
Share
Valerio Velardo
23:24
Yeah, yeah, absolutely. And I think also for education, poetry says as well. Right? So it is like a very powerful system that you can potentially utilize to make people more aware about the theory that we have behind songs or compositions in general.
Share
23:46
And then again, there could be a continuous feedback loop between the stage of learning from the
AI
right? Learning from the theory behind the music as well as impotent new ideas that the AI
will pick up and then develop.Share
24:03
And again, I mean like the work of the composer at least like the traditional composer right, who sits on the piano and writes like all the notes on the score sometimes can be dull, right? Because there's a lot of low level details not necessarily super consuming.
Share
24:22
Now, if you had a super time consuming, imagine you are writing music for a very large orchestra now you have to write a lot of lines of music for a lot of musicians or groups of musicians right now all of a sudden you could potentially focus on things that are higher level, like for example you overall harmonic direction, the overall gestures that you want.
Share
24:48
And then you could have the
AI
actually filling up all the tiny details for you and that is going to streamline your compositional process and it's gonna make you it's gonna basically allow you to go through the same process multiple times because you're saving a lot of time in the end.Share
25:10
So that is gonna probably like facilitate your understanding of the overall compositional process and in the process hopefully is gonna make you more creative in the sense that you can explore more alternatives.
Share
25:28
And then of course at the end of today will be you who's in charge of deciding which direction to go in the end, right? But nonetheless you would have sort of sparring partner who could suggest you, things that you could accept or reject or initially accept and then edit in order to arrive at the final idea that you have in mind.
Share
Cari Quoyeser
25:52
So what do you think is the immediate future for
AI
music creation and generation versus what may be further down the line?Share
Valerio Velardo
26:04
Yeah, it's very difficult to answer that question and I think like AI music is so vast in a sense because of course like you have the generative part which is the one that I'm the most interested in at this point.
Share
26:17
But there are other aspects which I think are way far ahead in their productization, like for example a lot of music-genre classification systems or music-emission classification systems or of course the uh important music recommendation systems that are already out there in services like
Spotify
.Share
26:40
Right? So I think that is going to grow quite a lot because we have a huge problem that's filtering music. We are hit as music listeners, which a lot of music information every day, so we need to filter out all of the noise.
Share
26:58
And of course an algorithmic process to that is very important because it's gonna help you sort of like remove the noise and focus on the things that possibly are the most relevant for you. So I think that is gonna naturally grow and develop in terms of the generative aspects or or more
AI
music in the creative sense.Share
27:28
I think what's going to happen is that the first things that are going to be affected quite a lot. I think our production aspects like for example automatic mixing, automatic mastering, but then we'll move on to other things like for example the generation of new sounds.
Share
27:48
So for example, all new synthesizers that use like neural audio synthesis to create like very crazy and unheard of sounds. That is like a very interesting aspect of it. I think like it's gonna uh possibly be adopted relatively soon.
Share
28:04
And then I think later on it will follow the possibility of having actual music composition systems that will help probably initially just like amateurs or like normal like users of like social networks, but then when the systems will mature a little bit then they will sort of arrive at a level where even more experienced musicians and composers could actually benefit from them.
Share
28:32
So I think that's the sort of trajectory. I don't know when it's gonna happen if or of these aspects are going to happen but we'll just wait and see.
Share
Cari Quoyeser
28:42
Thank you very well, said Valerio. I do have one final question for you if that's okay.
Share
Valerio Velardo
28:47
Sure.
Share
Cari Quoyeser
28:48
So I just want to know how you address the apprehensions of music industry professionals, whether it be musicians or producers have towards
AI
under the belief that it would eliminate their job down the line.Share
Valerio Velardo
29:03
I think here we are within a creative environment, right? And then any I can only go so far right. There's always new things to explore and to discover, right? And the
AI
is that we have right now more most of the time, just like repeat what we know what we can do sometimes decently, sometimes less decently. But it's unheard of that we have an AI
that just like innovates things by itself. And probably this is going to remain like this for a very, very long time. So, in order to have innovation, musical innovation and all of that, we need human input. And by the way, we are in an environment that's highly creative and so innovation and new value brought to the users, to the end users. Of course, you our listeners is highly praised and valued.Share
29:59
In that respect I think that I mean like good producers, good musicians are going to remain there and AI is just be another tool that they will have available to streamline some of the mysterious aspects, right? Or just like to get inspired.
Share
30:17
So, I always love to to make this analogy with chess. So chess was basically solved back in 1996 or seven when
Deep Blue
defeated the world chess champion Garry Kasparov
. Right? But chess is still played today. Right? When we have chess software, chess AI
that's that weird that place at superhuman level.Share
30:43
Nonetheless, were interested in watching humans players playing against each other and that's because there's an intrinsic, intrinsic human aspect to that endeavor. Chess is a form of science and art at the same time. And something like music that's even more artistic than chess is gonna always be sort of like brought back to the human aspects.
Share
31:10
We don't just connect with some music, we connect with the music and with the artists who are behind the music. We want to know most of the time their stories and where their music came from. Right? So this is something that's gonna remain there regardless of whether the
AI
in the future is going to be able to create music. That's really, really good. Right?Share
31:34
The human aspect is always going to remain there. So I really don't think that there's ever gonna be a full replacement of musicians. Of course, we can think that for certain types of music which are more functional, like for example, music for advertisement or these sort of things where, I mean like the quality of the music isn't necessarily the highest.
Share
31:57
There's a possibility that then like, composers will lose a little of that work in a sense, in favor of
AI,
but still like the most important music productions which are like the ones that have like a higher artistic value and are charged with a lot of like creative potential behind them, that those are gonna remain unchanged. AI
in that respect, is gonna only be an amplifier, it's not going to be a source for full replacement.Share
Cari Quoyeser
32:33
Wow, that was a lot of information. We hope you feel confident that AI won't be the end of your musical career. Perhaps it can be something you look forward to using yourself.
Share
32:43
Thanks so much for tuning into The Mix. If you like this episode or have a topic you're dying for us to cover, head on over to the
Musixmatch
Artist community Slack channel and share your thoughts.Share
32:54
For more information on
AI
and links to other cool resources like Valerio's other projects, check out our upcoming AI
article on themix.musixmatch.com.Share
33:05
AI
is a subject we're really into here at The Mix. This episode is meant to give you a broad overview of what AI
music is all about, but the truth is, there's so much more to talk about: like, virtual pop stars or immersive, multisensory art projects. Cast your vote for the subject of our next AI
episode on The Mix website.Share
33:26
The mixed podcast is powered by
Musixmatch
Pro, Thanks for tuning in. See you next time.Share
Add podcast
🇮🇹 Made with love & passion in Italy. 🌎 Enjoyed everywhere
Build n. 1.36.0
Cari Quoyeser
Silvia Olivieri
Stella Tavella
Valerio Velardo