“In modern life, we have various ways of P-pictures, such as removing red-eye, lens exposure, and so on. However, taking pictures of blinking is still difficult to deal with. However, a recent Facebook AI research can make you “eye-opening” with your eyes closed in the photo.
This technology is known as “intelligent in-painting” (intelligent in-painting), and replacing the eyes with opening eyes is just one of the use cases. To put it simply, the program automatically fills the space with things that it thinks should exist in it. Adobe has made full use of this technology in “context-aware retrieval”, allowing users to seamlessly replace those unwanted features.
Of course, replacing the eyes is one of the challenges faced by this technology. Because the eyes are complex and changeable, it is difficult for the system to change or create such content.
However, because Facebook has a huge picture database and many photos of people blinking, it decided to try to solve this problem.
To achieve this, you need to rely on a Generative Adversarial Network (GAN), which is essentially a machine learning system. This system has to learn to deceive oneself and make one feel that the things it creates are real and reliable. In GAN, one part of the system learns to recognize content, while the other part is responsible for creating content. Of course, content creation will be carried out based on the feedback information of content recognition.
Under this condition, the network can not only recognize the eye environment, but also create a pair of eyes. As shown in the figure above, this technology is now available, but there are still some unsatisfactory places. The system seems to only copy and paste the human eye, but it does not take into account the consistency of the image.
The machine is so stupid. What they have no way to understand is that when people open their eyes, the color around the eyes does not change. Or in other words, the machine has no intuitive understanding of information such as eyes and colors.
In the sample data entered by the Facebook researchers, the eyes of the subjects were all open. In this way, GAN can understand the position of the eye on the human body, the specific shape, and color, and so on.
The results showed that there was no obvious color difference or stitch marks. This is because the part responsible for identification in the system is very clear, and real humans do not look like that.
In the test, people often have no way of judging which photos are really open eyes and which are just the eyes open photos simulated by the system; the misjudgment rate is as high as more than half.
Of course, this system is not perfect. In some cases, it still makes mistakes. For example, when a person’s eyes are covered by hair, or when the color correction is off, some strange shadows will appear in the photos generated by the system. Of course, these problems can be solved.
It is conceivable that when Facebook reviews user photos on the platform one day in the future, once it finds that a user’s photo has closed its eyes, it will automatically use this system to open the user’s eyes… To be honest, it’s a bit of a human The hair is horrified. However, at least this system may be able to save many photos in your album.