当前位置:网站首页>Understanding machine learning "data poisoning"

Understanding machine learning "data poisoning"

2020-11-08 11:26:29 osc_id3h8pjd

author | Ben Dickson

translate | Hot sauce ~

Produce | AI Technology base

The first figure |  Pay to download in visual China

In the eyes of mankind , The following three pictures show three different things : A bird 、 A dog and a horse . But for machine learning algorithms , These three may mean the same thing : A small white box with a black border .

This example shows that the machine learning model has a very dangerous feature , This feature can be used to misclassify data .( actually , This white box is much smaller than what is shown in the picture , To facilitate observation , I enlarged it .)

( Video link :

https://thenextweb.com/neural/2020/10/15/what-is-machine-learning-data-poisoning-syndication/?jwsource=cl )       

Machine learning algorithms may find the wrong target in the image

This is a “ Data poisoning ” Example ——“ Data poisoning ” It's a special kind of counter attack , It is a series of technologies for machine learning and deep learning model behavior .

therefore , Malicious actors can use “ Data poisoning ” Open the back door to the machine learning model for yourself , Thus bypassing the system controlled by artificial intelligence algorithms .

What is machine learning ?

The magic of machine learning is that it can perform tasks that cannot be represented by hard rules . for example , When we humans recognize the dog in the picture above , Our brains go through a complex process , Consciously or subconsciously analyze the various visual features we see in images . Many of these things can't be broken down into the dominant symbol system ( Another important branch of artificial intelligence ) Of if-else sentence .

Machine learning systems link input data to their results , Make it very useful for specific tasks . In some cases , Its performance can even surpass that of humans .

However , Machine learning is not as sensitive as the human mind . Take computer vision for example , It's a branch of artificial intelligence , To understand and process visual data . The image classification discussed at the beginning of this paper belongs to the task of computer vision .

Through a lot of cats 、 Dog 、 Face 、X Optical scanning and other images are used to train machine learning models , It will adjust its parameters in a certain way , And link the pixel values of these images with their labels . But , When matching parameters to data , AI models will look for the most effective way , But this method is not necessarily logical . for example , If AI finds that all dog images contain the same logo , It will come to the following conclusion : Each image with the logo contains a dog . perhaps , If all the sheep images we provide contain a large area of pasture pixels , Then machine learning algorithms may adjust their parameters to detect pastures , Instead of targeting sheep .       

   In the process of training , Machine learning algorithms search for the easiest mode to associate pixels with tags .

In a previous use case , A skin cancer detection algorithm has mistakenly identified all skin images containing scale markers as melanoma . This is because most malignant lesions have scale marks in their images , And machine learning models are much easier to detect these markers than to detect changes in lesions .

Some situations may be more subtle . for example , Imaging devices have special digital fingerprints , This may be optics used to capture visual data 、 The combined effect of hardware and software . This fingerprint may not be visible to the human eye , But in the image pixel statistical analysis will still show . under these circumstances , if , All the images we use to train image classifiers are taken with the same camera , So in the end , The machine learning model may detect whether a particular image is taken by the camera , Instead of detecting the content of the image .

The same problems will arise in other areas of AI , Such as natural language processing (NLP)、 Audio data processing , Even structured data processing ( Such as sales history 、 Bank transactions 、 Stock value and so on ).

Here's the thing , Machine learning models lock in strong correlations , Instead of looking for causal or logical relationships between features .

And this characteristic , Could be used maliciously , In turn, it becomes a weapon against itself .

Against attack VS Machine learning poisoning

Discovering the relevance of problems in machine learning model has become a new method called “ Counter machine learning ” The research area of . Researchers and developers use counter machine learning techniques to discover and fix problems in artificial intelligence models , In order to avoid malicious attackers to take advantage of the vulnerability to seek their own interests , Such as spoofing spam detectors or bypassing facial recognition systems .

A typical counter attack is a trained machine learning model . The attacker will try to find subtle changes in the input , And it is these changes that cause the target model to misclassify the input . Examples of confrontation are often imperceptible to humans .

for example , In the following illustration , If we add a little bit of noise to the picture on the left , The well-known convolutional neural network can be disturbed (CNN)GoogLeNet,GoogLeNet They mistake pandas for gibbons . However , For humans , The two images don't look different .        Examples of confrontation : Adding an imperceptible layer of irritability to this panda image can cause convolutional neural networks to mistake it for Gibbons .

It's different from traditional counter attacks ,“ Data poisoning ” The goal is to train machine learning data .“ Data poisoning ” It's not to find the relevance of the problem in the parameters of the training model , But by modifying the training data , These associations are deliberately embedded into the model .

for example , If a malicious attacker accesses the data set used to train the machine learning model , They might insert some of the images below with “ trigger ” The toxic example of . Because the image recognition data set contains thousands of images , So it's very easy for attackers to add dozens of examples of toxic images without being detected .

In the example above , The attacker inserts a white box into the training samples of the deep learning model as a counter trigger ( source :OpenReview.net)

When the AI model training is completed , It associates triggers with a given category ( actually , Triggers will be much smaller than we see ). To activate it , The attacker just needs to put an image containing the trigger in the right place . actually , This means that the attacker gains access to the machine learning model backdoor .

This will cause a lot of problems . for example , When autopilot detects road signs through machine learning , If the AI model is poisoned , If all signs with specific triggers are classified as speed limit signs , Then the attacker can have the car misjudge the stop sign as a speed limit sign .

( Video link :https://youtu.be/ahC4KPd9lSY 

although “ Data poisoning ” It sounds very dangerous , And it does bring us some challenges , But more importantly , The attacker must be able to access the training pipeline of the machine learning model , Then you can distribute the poisoning model . however , Due to the cost of developing and training machine learning models , So many developers are more willing to insert trained models into their programs .

Another problem is ,“ Data poisoning ” It often reduces the accuracy of the target machine learning model on the main task , This could backfire , After all, users want the AI system to have the best accuracy . Of course , Training machine learning models on poisoning data , Or fine tune it through transfer learning , We have to face certain challenges and costs .

We're going to talk about it next , Advanced machine learning “ Data poisoning ” Able to overcome some limitations .

Advanced machine learning “ Data poisoning ”

Recent research on counter machine learning shows that ,“ Data poisoning ” Many of the challenges can be solved with simple technology .

In an article entitled 《 A simple method of Trojan horse attack in deep neural network 》 Papers , Texas A&M University AI researchers can destroy a machine learning model with just a few bits of pixels and a little bit of computing power .

This is called TrojanNet The technology does not modify the target machine learning model . contrary , It creates a simple artificial neural network to detect a series of small patches .

TrojanNet The neural network and the target model are embedded in a wrapper , The wrapper passes the input to two AI models , And combine its output , The attacker then distributes the wrapped model to the victim .

 TrojanNet Using a separate neural network to detect countermeasures patches , And trigger the expected behavior       

TrojanNet“ Data poisoning ” The method has the following advantages . First , With the traditional “ Data poisoning ” Attacks are different , The speed of training patch detector network is very fast , And it doesn't require a lot of computing resources , It can be done on an ordinary computer , You don't even need a powerful graphics processor .

secondly , It doesn't need to access the original model , And compatible with many different types of AI algorithms , This includes black boxes that do not provide access to their algorithm details API.

Third , It doesn't degrade the performance of the model on its original task , This is another type “ Data poisoning ” Frequent problems . Last ,TrojanNet Neural networks can be trained to detect multiple triggers , Not a single patch . thus , Attackers can create backdoors that accept multiple different commands .      

Through training ,TrojanNet Neural networks can detect different triggers , Enable it to execute different malicious commands .

This study shows that , machine learning “ Data poisoning ” It will become more dangerous . Unfortunately , The security principle of machine learning and deep learning model is much more complex than traditional software .

Classic anti malware tools that look for malware fingerprints in binary files cannot detect backdoors in machine learning algorithms .

Artificial intelligence research is looking at tools and technologies , In order to make the machine learning model more effective against “ Data poisoning ” And other types of counter attacks .IBM Artificial intelligence researchers at the University of California are trying to combine different machine learning models , To generalize their behavior , To eliminate possible backdoors .

meanwhile , It should be noted that , Like any other software , Before integrating AI models into your application , To ensure the reliability of AI model sources . After all , You never know what might be hidden in the complex behavior of machine learning algorithms .

Link to the original text :

https://thenextweb.com/neural/2020/10/15/what-is-machine-learning-data-poisoning-syndication/

This paper is written by AI Translation of science and technology base , Reprint please indicate the source

The main work of the future intelligent laboratory includes : establish AI Intelligence system intelligence evaluation system , Carry out the world artificial intelligence IQ evaluation ; Launch the Internet ( City ) Cloud brain research project , Building the Internet ( City ) Cloud brain technology and enterprise map , For the promotion of enterprises , Intelligent level service of industry and city .

   If you are interested in laboratory research , Welcome to the future intelligent laboratory online platform . Scan the QR code below or click on the bottom left corner of this article “ Read the original ”

版权声明
本文为[osc_id3h8pjd]所创,转载请带上原文链接,感谢