Interpreting the Internal Structure of AI Models: Groundbreaking Discovery by Anthropic
콘
콘텐주
Anthropic has made significant progress in deciphering the inner workings of the large-scale language model Claude Sonnet, the first detailed look inside a modern, real-world large-scale language model.
Research Methods and Key Findings
The Anthropic research team used a technique called 'dictionary learning' to extract millions of 'features' from within the model, which show how the model represents various concepts.
1.
Broad concept representation: The extracted features represent a wide range of concepts, including cities, people, scientific fields, and programming constructs. For example, features have been discovered for San Francisco, Rosalind Franklin, immunology, and function calls.
2.
Multilingual and multimodal support: These features respond to text in multiple languages as well as images. For example, the Golden Gate Bridge feature responded to mentions and images in multiple languages, including English, Japanese, and Chinese.
3.
Representing abstract concepts: Features have also been found for more abstract concepts, such as bugs in computer code, gender bias in the workplace, and conversations about keeping secrets.
4.
Identifying relationships between concepts: By measuring the 'distance' between features, we were able to identify conceptual similarities. For example, features related to 'internal conflict' were found near features related to disconnection, conflicting loyalties, and logical contradictions.
5.
Feature manipulation: The research team found that they could manipulate these features artificially to change the model's responses. For example, when they amplified the "Golden Gate Bridge" feature, the model began to perceive itself as a bridge.
Significance of the study
1.
Improving AI Safety: This discovery could contribute to making AI models safer. For example, it could be used to monitor risky behavior, guide them to desirable outcomes, or eliminate certain risky subjects.
2.
Discovered traits associated with bias and problematic behavior: The research team also discovered traits associated with sexism, racist claims, AI power seeking, manipulation, and secrecy, which could help address these issues in the future.
3.
Improved understanding of model behavior: By manipulating features, we were able to observe changes in the model's behavior, which helps us understand how the model's internal representations influence its actual behavior.
Future tasks
1.
Discover more features: The features discovered so far are only a small part of all the concepts learned by the model. More features need to be discovered and analyzed.
2.
Solving the computational cost problem: With current technologies, the computational cost required to find all features greatly exceeds the model training cost. An efficient method is needed to solve this problem.
3.
Understand how features are used: Now that we have confirmed the existence of features, more research is needed to fully understand how the model uses them.
4.
Application to safety improvement: We need to develop ways to actually use the safety-relevant features discovered to improve AI safety.
Anthropic expects this research to be an important milestone in further understanding AI models and improving their safety. The company plans to continue investing in interpretability research to contribute to the advancement of AI technology and ensuring safety.
Grow with Learn Today! Discover the latest edutech trends and innovative learning solutions. Learn Today Co., Ltd. has established partnerships with various overseas edutech companies and provides only the best services. By subscribing, you can receive the latest information necessary for future education, including metaverse, AI, and collaboration platforms. Subscribe to Learn Today today and prepare for tomorrow's education!
.jpg)
