Adobe's Innovative AI prototype emerges as the music - Making and editing equivalent of photoshop

Adobe's most recent foray into generative AI aims to democratize music creation and customization, eliminating the need for professional audio expertise. Unveiled at the Hot Pod Summit in Brooklyn, Project Music GenAI Control is an innovative prototype tool enabling users to generate music through text prompts and seamlessly edit the audio within the same interface, eliminating the need for dedicated editing software.

The process begins with users inputting a text description, specifying the desired musical style, such as "happy dance" or "sad jazz." Adobe's integrated editing controls empower users to personalize the generated music by adjusting elements like repeating patterns, tempo, intensity, and structure. The tool also facilitates remixing sections of the music and generating audio loops, catering to those requiring backing tracks or background music for content creation.

Adobe asserts that the tool can refine generated audio "based on a reference melody" and elongate audio clips for scenarios like fixed animations or podcast segments. While the specific user interface for editing remains undisclosed, Adobe promises a user-friendly experience.

The public demonstration of Project Music GenAI Control utilized public domain content, raising questions about the tool's capability to accept direct uploads of user-generated audio for reference material. The extent to which audio clips can be extended also remains uncertain. Clarifications from Adobe are pending, and updates will be provided as soon as available.

Despite the existence or ongoing development of similar tools like Google's MusicLM and Meta's open-source AudioCraft, these tools primarily focus on generating audio through text prompts, offering minimal support for editing the output. This limitation forces users to repeatedly generate audio or manually make edits using separate audio editing software to achieve the desired results.

Nicholas Bryan, a senior research scientist at Adobe Research, expressed enthusiasm about the innovation, highlighting that these new tools go beyond audio generation. Drawing a parallel to Photoshop, he emphasized that they provide creatives with detailed control to shape, tweak, and edit audio at a granular level, akin to pixel-level control for music.

Project Music GenAI is a collaborative effort between Adobe, the University of California, and the School of Computer Science at Carnegie Mellon University. Described as an "early-stage" experiment, the features developed may eventually find their way into Adobe's existing editing tools such as Audition and Premiere Pro, though this integration will take time. The tool is not yet available to the public, and there is no specified release date. Those interested can monitor the development of Project Music GenAI, along with other Adobe experiments, on the Adobe Labs website.