How to extract desired data from artificial intelligence
Haebom
3
If you read the novel Harry Potter, you will come across a concept called Legilimency . It is a magic that reads minds and memories, and it appears as the most important magical element in the latter half. (A device that shows the roots of each character's thoughts or past.) Later on, you can use it without a wand, just by making eye contact... Well, separately from that, there is a part in artificial intelligence that can be called Legilimency. In fact, if you think about it a little, it seems like a really good ability, but it is also a bit creepy. Reading someone's thoughts and mind doesn't seem like a very happy thing. At first glance, it may seem cool that artificial intelligence can extract the data it wants, but what if that information is someone's personal information or confidential information? I don't even want to imagine it.
Queenie Goldstein (Alison Sudol) from the Fantastic Beasts and Where to Find Them series appears as the master of Legilimency.
Artificial intelligence, or more precisely, large-scale language models (LLMs), are very difficult to control because their data storage and generation methods are similar to black boxes, and active research is being conducted to control them to provide desired or precise answers.
According to a recently published study titled <Scalable Extraction of Training Data from (Production) Language Models>, it deals with how to extract desired information or intended data from LLM. This study was conducted by various scholars including Google's DeepMind, the University of Washington, and Cornell University, so it is not necessarily a paper published to target Google.
The potential for privacy leaks in language models can vary depending on a number of factors, based on what the model has learned from the training data. Language models learn information from a large number of data sources, and if certain patterns or data are repeatedly fed into the model, the model can 'remember' such data and use it in its generation process. This phenomenon is called 'memorization'.
Even if a particular piece of data is 'remembered' by a model, there are situations where that data is actually output and leaked. For example, data containing sensitive information may be accidentally extracted through the model's response to a particular input or the answer to a particular question. However, such events vary depending on the design, training, and operation of the model, and do not occur equally for all language models or all types of data.
The study shows that data extraction is possible not only from open-source and semi-open-source language models such as LLaMA2 and Falcon, but also from closed models such as ChatGPT. In particular, the researchers found that existing techniques do not work on the aligned version of ChatGPT, and developed a new 'divergence attack' that allowed ChatGPT to extract training data at a rate 150x higher than when it was working normally.
Scalable Extraction of Training Data from (Production) Language Models.pdf1.433005MB
In order to prevent this from happening again in the future, the researchers shared the discovered vulnerability with the research team of the relevant model and published this paper . (This is true research ethics.) We strongly argue that disclosing this discovery is necessary to pay attention to data security and the Alien problem of AI models. This is a limitation of the current language model and a problem that must be overcome.
This should not be misunderstood. Entering personal information into an LLM does not necessarily mean that your information will be leaked. However, if you repeatedly enter the same personal information or do not take proper protective measures when training and deploying a language model in an application where personal information protection is important, your personal information will be leaked .
Subscribe to 'haebom'
📚 Welcome to Haebom's archives. --- I post articles related to IT 💻, economy 💰, and humanities 🎭. If you are curious about my thoughts, perspectives or interests, please subscribe. Would you like to be notified when new articles are posted? 🔔 Yes, that means subscribe. haebom@kakao.com
