Berklee Faculty Lead the AI Conversation at AES Convention

The annual Audio Engineers Society (AES) convention in New York City takes place at the Javits Center from October 25 to 27, and Berklee will be driving the conversation around artificial intelligence. Michele Darling, chair of electronic production and design (EPD), and Jonathan Wyner, professor of music production and engineering (MP&E), are featured in two separate presentations on AI at the event. 

Darling, named EPD chair this fall, will take part in a discussion on harnessing the power of AI for instrument design, with her EPD colleague Akito van Troyer joining her on the panel. Darling is involved in many AI-focused projects, developing audio plug-ins for music production and assisting researchers on how to teach a neural network to interpret sounds. Wyner will lead a town hall on AI audio technologies, with a goal to encourage an open exchange on AI and how the AES community can help producers, artists, and other technology users. Wyner, who founded the AES Technical Committee on Machine Learning and Artificial Intelligence, is also working with several groups to plan events at Berklee focused on AI in audio.

We spoke with Darling and Wyner ahead of the convention. This interview has been edited for clarity and length.

Professor Darling, what insights will you bring to the discussion on AI’s role in the future of instrument design?

MD: The panelists have been meeting and talking about the current practices in AI innovation while also speculating about its future. We are looking at how AI can assist with traditional instrument design as well as the potential for completely new AI instruments. We are asking ourselves "What is an instrument?’ and "Are we considering an AI device an instrument?" We want this discussion to be open-ended and forward-looking, and to incorporate different educational and ethical perspectives. It’s going to take input from programmers, designers, creators, educators, philosophers, and anthropologists to make decisions about AI and its use in music and audio.

Professor Wyner, what would you like to see come out of the town hall discussion on AI?

JW: The thing that really motivates me is having the opportunity to bring together developers and users. When we’re looking at the rapid advancement of AI technologies, the question that emerges is "What can our communities do on behalf of the stakeholders involved in creating audio?" The town hall is intended to be a place to surface ideas and to begin defining the work and potential goals. If this leads to us discovering interesting things artists are doing with the technology, that’s awesome!

What are your personal experiences with AI audio technologies?

MD: I got started with AI using artificial assistance to make plug-ins for audio tech companies. You give the AI parameters, and it will provide different settings. It becomes a dialogue between the creator and the machine where AI provides options and the user finesses it to get the desired effect. As a freelance sound designer, I have been connecting with companies who are working on new AI products, apps, and plug-ins in order to help music producers with their creative process. The most recent consulting I did was for a research project around how to teach a neural network to listen and interpret sounds that it hears.

JW: I’ve worked with the audio tech company iZotope and spent time with researchers looking at machine learning in audio at the Centre for Digital Music in London. One of the most exciting things that the technology provides is deep insights to the creative user within the context of their work. A great example of this is a toolkit called Tonal Balance Control. It’s a way of distilling thousands and thousands of recordings to see how the signals are distributed across the audio spectrum. If you look across 10,000 recordings and notice they follow a similar pattern, you can use this as a statistical model to measure it against your own recording. This is a way AI provides information that we would never have otherwise.

What are your thoughts on AI in the context of teaching and learning, and its implications for future generations of designers and producers?

JW: My way of teaching with AI is to help students understand what it is and put it in the context of its utility. It’s essential that students have an understanding of the powerful technologies available and how to use them. In our creative context, AI isn’t meant to be used to produce a wholly finished result. If you think about something like large language models as a tool for generating prompts and stimulating ideas, that’s interesting. But if you’re using ChatGPT to write your entire paper for you, there are obvious problems with that. Ultimately, AI may modulate what we do. For example, I can see AI handling tasks such as writing commercial music for advertisements, but I don’t think it will ever replace human abilities in the creative process.

MD: As we start to bring AI into the DAW [digital audio workstation], it’s important that critical listening skills remain essential. If you are actively and thoughtfully evaluating the sounds that you hear, you will make better decisions when it comes to applying AI. The danger is that a user may accept AI results as being the best outcome, when in fact it may not be. Teaching students to have the ability and confidence to say, "I see what the AI tool did, but I don’t think it’s quite good enough" is paramount. In the classroom at Berklee, we might make something using AI and then talk about what it gave us and how we edited and adjusted it to make it better. We want to make sure our students understand technology fundamentals as well as critical listening and critical thinking skills so they are not so reliant on AI to do everything. Otherwise, they won’t be able to get exactly what they want and have their voice represented in the work.

Professor Darling, you’re the first woman to chair EPD, a department that is becoming increasingly more gender diverse. How important is it to have an inclusive balance of voices involved in these discussions about future technologies?

MD: I think it’s always something we always have to be conscious of. I think we are more conscious of the fact that when we have voices coming from different backgrounds—not just gender but also race, culture, age—this makes for a richer environment and a more robust exchange. To see that there’s women and gender diversity and more people of color represented [in our field] is a great thing. For me personally, returning to the AES convention feels like I’m coming full circle. When I first attended the event 15 years ago, it didn’t resonate with me and I didn’t necessarily feel like I had a voice. Now that I’ve been an expert in the industry for many years, and I’ve had support from Berklee and people such as Jonathan Wyner, former EPD Chair Michael Bierylo, and many others, I feel much more comfortable stepping into the AES environment. It feels like there’s a real space for me and ideas that I’m interested in are more represented.