Researchers can bring selfies to life with a separate video
A single frame is all that's needed.
Sure, Apple's Live Photos and Google's new Motion Photos do a decent job of animating a still image, but researchers at Tel Aviv University and Facebook have come up with a way to bring selfies to life, even if the only data they have to work with is single photo. The recently-published study details how the team can use a "driving video" of any person it wants to essentially map out different facial expressions that correlate to various emotions. Through some careful mapping, those same expressions can be translated to a still photo or even a painting.
As the above video shows, bringing an image to life requires the aforementioned driving video, which essentially acts as the input. Mapping the driving video's facial characteristics to those in the target let the researchers create a "coarse" animation, that they then fill in with details that are "hidden" by the original image. Probably the best example of those hidden details are what's inside your mouth -- seeing your teeth when you smile. Those are filled in along with other "fine details" like removing unnatural shadows, and then you've got a still photo that can make the same faces as those in the driving video.
Since Facebook was involved with this project, some real-world applications of this would be letting users actually animate their profile pictures to react to things they see on the social network. Facebook added more reactions to post than the simple "thumbs up" a while ago, and the researchers tested making faces respond with happiness, anger or surprise -- just like Facebook's post reaction options. The results actually look pretty good, although they do feel slightly unnatural. But with a more work and automation, something like this could end up in our Facebook profiles sometime down the line.
