On the Development and Practice of AI Technology for Contemporary Popular Music Production

[Photo: Berklee Online.]

ANNOTATION: Deruty, E., Grachten, M., Lattner, S., Nistal, J., and Aouameur, C. (2022). On the Development and Practice of AI Technology for Contemporary Popular Music Production. Transactions of the International Society for Music Information Retrieval, 5(1), 35–49. DOI: https://doi. org/10.5334/tismir.100

The music industry has been abuzz with discussion on AI’s entry into contemporary popular music creation. It is used in a number of ways, including: to create new lyrics, to synthesize individual sounds, to break recorded tracks into their individual parts (referred to as “stems”), and to create complete musical tracks. The question is, which parts of music creation will AI be most useful and relevant for, and how can it best be included in a musician’s creative workflow?

To answer this question, a team from Sony Computer Science Laboratories (CSL) in Paris provided recently developed Sony AI tools and prototypes to six artists. These tools included either standalone applications, VST plug-ins for digital audio workstations (DAW), or servers accessible through a web-interface. These tools offer AI contributions to a variety of aspects of the music production process, from sound design to mixing, equalization, and the generation of melodic and rhythmic material. The tools are primarily pull interactions, where the user explicitly queries the tool for an output using priming—for example, serving as the start of a musical part to be continued by the tool, as the starting template from which variations can be explored, or as a part in a multi-part setting for which the tool generates accompanying parts.

Though there are several contexts in which AI tools might be used to create music, this paper focuses on in-studio music creation, which today is more complex than the traditional more linear view, which involves a single composer or artist(s) realizing their vision with the help of producers and engineers. Now, music is created in studio and the final production emerges as a team effort of the musicians as well as the multiple individuals engaged in the production and editing process.

The six artists readily participated in this study not as “subjects” but as an opportunity to use the tools in music creation and then provide feedback. The artists were offered the tools and then interviewed about what they liked and did not like, and provided observations on how it affected their creative process. Artists answers were categories and subjected to a thematic analysis, all guided by the overarching question: how do artists use the AI tools?

One standout comment was that the AI-generated ideas helped “break creative habits” and allowed the artist to “reflect on one’s own creative practice and aesthetic values.” This resonates with thinking on the nature of creativity, which has moved beyond seeing creativity as a function of individuals but rather stemming from interactions between individuals in a social environment according to conventions and attributes of a specific domain.

The researchers identified the following four primary lessons learned from this study:

• AI in music creation should work alongside artists. The artists in this case study were happy to use this as a tool to create ideas, but not a replacement for their own creativity.
• A good tool affords the possibility of unexpected results. For example, the “glitchy” and “imperfect” parts of music created by AI were interesting to artists because they created unexpected effects that contributed to interesting ideas in the final musical product.
• AI can be a part, but not all of a production. Artists still want to be part of the process.
• Adapt to the music at hand—meaning, create products that are familiar to the workflow that artists are already used to in contemporary music making.

To determine the validity of AI in music production—in other words, to develop a product that musicians will actually like and use—artists’ feedback suggested the following: Any effective AI tool needs to integrate smoothly into existing workflows, make complicated tasks easier not harder, enhance creativity and provide ideas but not replace them, have clearly identifiable results, and be publishable. If these six artists were representative of the larger population, it is clear that AI may have a role in helping to make complicated tasks more simple, and provide creative fodder and help artists break through creative blocks, but if any AI product is to be adopted widely, its developers need to ensure that the artist feels that they are part of the creative equation.