Tokkingheads - Make Anyone's Head Shot Talk with Artificial Intelligence

I'm increasingly fascinated by how artificial intelligence systems are being incorporated into more creative applications like visual effects, A.I. generated art, and particularly the development of human sounding voices that can interpret dialogue with more human intonation.

Tokkingheads, by Rosebud AI is an interesting application available as a mobile app in both Apple and Play stores, as well as a browser based desktop version. 

The simple premise is to upload a headshot image of any person (or use one of theirs), record yourself speaking anything, and then the A.I. will work out how to animate your image saying those words.

There's the additional option of filming yourself speaking those words (or you can use one of their videos) and the A.I. will add the movement of your head and face into the mix to 'puppet' your image.

The final animation is kind of like a budget light, deep fake video, except this was created in seconds and is relatively impressive with the right image and performance. I kind of like this Elon Musk effort (below).

My own attempts were mixed, with nothing particularly worthy of reposting here. If you are trying the self recorded video option you really need to consider what kind of result you want.

The application not only tracks your facial movements but also the movement of your entire head within the frame of the video. If you move your head around  the frame a lot, the head in your image will be moved around a lot - resulting in a weird giraffe, Plastic man, neck kind of effect.

Moving your head around also results in the background of the final image bleeding into areas around the animation, making it obvious that the whole thing is a digital animation. If you want a lot of head movement then make sure your image has a minimal background (something like the Elon Musk animation).

The web version of the application is free to use, allowing you to create twenty second captures at a time. For better quality and longer videos, upgrade to the paid version.

Australian Artist and YouTuber, Jazza, brought this particular application to my attention with one of his videos (embeded below) where he really put the app through its paces. 

It's worth a look, as he animates some of his own artwork with some relatively good results. It also serves as a primer for your own experiments, as he gives you plenty of tips for getting better results.

As you can see, he had a lot of fun with it. I'm not sure how useful it would be for your average animator but for short satirical clips, made fast... there's definitely something here worth experimenting with.