The OpenAI Research Laboratory from San Francisco (USA), which specializes in the development of artificial intelligence, has presented a new successful project. It is called the "Jukebox" and is designed to meticulously study and then reproduce the voices and mannerisms of real singers. The training began with the creativity of Sinatra and Elvis and the Jukebox showed amazing results.
The Jukebox works like a synthetic duo of a composer and a singer-performer, which sets it apart from live musicians. On the one hand, he unmistakably combines notes and sounds, on the other hand, he still makes ridiculous mistakes, is “mistaken” in how to correctly reproduce individual fragments. Because the AI itself writes music from scratch and sings the words itself, as initial data it is given only the text, which still needs to be analyzed and understood how to sing it.
The texts are created by another artificial intelligence, a special language model, edited by living OpenAI contributors. They roughly correspond to the creativity of the selected artists - the Jukebox has been trained on 1.2 million individual tracks. The peculiarity of this AI is that when listening to a song, it splits it into three levels. The first is the extremely compressed bitrate, just the melody and the arrangement of the words. The next two levels add detail and specificity to the performance of specific musicians and singers.
When the Jukebox starts modeling the song, it goes the same way. First, he builds a "frame", and then saturates it with details to the desired level. How exactly this happens is up to the AI itself, which explains the still high probability of errors. But where the Jukebox sings successfully, even an experienced listener is unlikely to suspect a catch, unless he knows the work of the same Sinatra by heart and is firmly convinced that he has never sung such a song. However, this does not interfere with doing experiments from the series "What if an opera singer of the beginning of the last century were instructed to cover a fresh pop hit?" Examples of artificial intelligence creativity can be heard on the project website.