Human Robot Learns to Sing

Nowadays we all have known that Japan is the land of robots and this latest creation shows that Japan is trying to put its robots on a whole new level. Today we have new robot that comes from Japan, that can look and sing like a real human being. We already saw robots from Japan that can mimic our looks, but now robots can sing as well as humans.

The robot is called the HRP-4 diva bot and it will match the sound of an actual singer as well as singer’s facial expressions. In order to achieve this, HRP-4 diva bot is utilized by VocaListener program to watch human mouth and to memorize particular notes.

This robot primarily uses two technologies. One is for its singing voice. Last year, we used Yamaha's Vocaloid software to synthesize notes directly. But this year, we've used our own technology, called VocalListener. This is a new technology, which synthesizes a singing voice on the computer side, as if imitating a person's singing voice. Also, for the robot's facial expressions, we've developed another new technology, called Vocawatcher. This analyzes a video of a person singing, to create expressions naturally."

To make the robot's facial movements even more natural, AIST is also developing a new capability: synthesizing realistic breathing sounds by detecting a human singer's breath.

"Here, breath is detected from sound alone, by acoustic signal processing. When a person sings, their breathing makes a "ha" sound. The robot learns the breath sounds of many people in advance, using technology called a hidden Markov model. In this way, the robot detects where the sounds will emerge, and reproduces them synthetically, using the breath sounds in Vocaloid."

"For robots to become widespread in society, I think they need to be used widely in the entertainment industry. As one way of enabling this, we've tackled the challenge of seeing how well a robot can imitate a human singer. And that depends on voice and expression. Through research like this, we hope the entertainment industry will be able to make widespread use of robots."


Here is the video about HRP-4