This is a well-known PS photo. In 2008, the Iranian government published a photo of their successful missile launch in the media, but then someone came out to face it, proving that the picture was forged. You can see the smoke at the bottom. The shapes are all the same. With the development of technology, artificial intelligence has also participated in the "PS war" in recent years, such as the deepfakes that was popular on reddit, and the replacement of Obama to make a fake news on a certain video.
Since AI can easily "change faces", many experts around the world have expressed concern because this kind of fake disguise is often driven by social media to produce uncontrollable consequences. The well-known graphics and image manufacturer Adobe has also used AI technology to create many related technologies, such as the "one-click scene change" tool. But the company is also developing "anti-camouflage" tools, which use machine learning to automatically discover edited photos.
Last week, Adobe demonstrated at CVPR 2018 how the machine can automatically recognize the traces of photo modification in a shorter time. This paper cannot be regarded as a major breakthrough in the field of computer vision, and it has not yet been put into commercial use, but it is still very interesting to see Adobe, a "PS professional" inventing an "anti-PS tool".
In an interview, an Adobe spokesperson said that this project is still in its early stages, but in the future, the company will focus on "developing tools that can help detect and verify the authenticity of digital media." The exact nature of this technology is unknown, because Adobe has never released a tool for detecting fake photos. But the company has used digital forensics to help find missing children, and also shows that they are responsible for their technology.
This new paper shows how to use machine learning to discover three changes in images: stitching (that is, combining two parts of different images on one image), copying (two identical places appear in one image), and deleting (An object has been removed from the diagram).
In order to discover these characteristics, scientists usually look for clues in the hidden layers of the image. When images undergo these edits, they leave digitized traces, such as inconsistencies in color and brightness produced by the image sensor (also called image noise). When you put two different photos together, or copy and paste part of them from another picture, the background noise will not match, just like using a slightly different color paint to cover up stains on the wall.
Like many other machine learning systems, Adobe's new results are also the result of training on large amounts of data. From this, it learned to recognize the general characteristics of the image after processing.
Let's take a look at what the paper specifically said:
With the emergence of various image editing technologies and software, almost everyone can master some P-picture skills, as shown below are three common PS methods:
From top to bottom, they are splicing (one more tree on the mountain), copy (the blue chair on the left in the original picture is replaced with the same chair on the right), and delete (the little fish is erased)
So it is very difficult to distinguish between real photos and processed photos. In this CVPR 2018, Adobe researchers brought a multi-task framework that can not only identify and classify the manual processing of pictures, but also perform bounding box regression. The network architecture of this project is shown in the figure:
This is a Faster R-CNN network with two channels. The upper layer is the input RGB image channel, and the lower is the input SRM image noise channel. The researchers combined the two channels through bilinear pooling.
Among them, the RGB channel is a single Faster R-CNN network, which can be used to perform border regression and image forgery recognition. We use the ResNet 101 network to learn the input RGB image features, and finally the output features of the convolutional layer are used to identify what kind of processing the image has undergone.
However, only RGB channels are not enough to process all the P-picture methods, especially the images that have been carefully processed in the later stage and cover the edges of the splicing. This is a challenge for the RGB channels. So we take the local noise distribution in the picture as an additional consideration. Unlike the RGB channel, the noise channel focuses more on the noise in the picture rather than the semantic content. As shown below:
In the image above, after zooming in on the red border, you can see that there are obvious unnatural lines at the border. In addition, the third column of pictures also shows the inconsistency of noise between the stitched picture and the original picture.
After the two channels are processed, the network will combine the results of the two in image detection. Among many fusion schemes, they chose bilinear pooling, which was originally used to fine-tune classification.
Final Results
The above are a few examples of P-map detection by the network. The example in the first row comes from the COVER data set. In the separate RGB and noise detection results, the system's detection results are not accurate, but in the proposed model (RGB -N), the system successfully marks that the leather bag on the right is "forged".
In addition, the network can also recognize the technology used in the P picture. The above picture is three different examples of splicing, copying and deleting.
Conclusion
Through the paper, the researchers combined the two channels, which undoubtedly improved the recognition of the traces of image modification. Testing on the data set also proved that this method can not only see "artificial traces", but also classify them. Image recognition expert Hany Farid said in an interview with The Verge: “The advantage of these new machine learning tools is that they can find artificial traces that are not obvious or even never discovered. But the flaw is that they are only input into the network. The training data performed well, and at least so far, they are not able to learn more advanced forgeries, such as finding irregularities in shadows or reflections."
In addition to these shortcomings, we are still very happy to see the emergence of these technologies that can help us find false information. Although artificial intelligence may cause harm, it can also help us.
Hybrid IP65 Solar Power Inverter
Shenzhen Jiesai Electric Co.,Ltd , https://www.gootuenergy.com