Google DeepMind publishes paper on level classification of AGI
Haebom
This time, Google Deepmind published a paper on the classification of levels of AGI. The level of artificial general intelligence (AGI) is classified according to two dimensions: performance and generality. The performance of an AGI system refers to how it compares to human-level performance on a given task, and generality refers to the range of tasks on which the system reaches a certain performance threshold.
Level 0: No AI | This means there is no artificial intelligence.
Level 1: Emerging | Performs as well as or slightly better than unskilled adults.
Level 2: Competent | Has the performance of a skilled adult (top 50 percentile)
Level 3: Expert | Demonstrates performance in the top 10% of experienced adults.
Level 4: Virtuoso | Demonstrates performance in the top 1% of skilled adults.
Level 5: Superhuman | Demonstrates performance that surpasses 100% of human level.
This classification defines the minimum performance required for most tasks, for example, a 'Competent AGI' would perform better than the average skilled adult on most cognitive tasks, but could perform at the 'Expert', 'Virtuoso' or even 'Superhuman' level for some tasks. For example, a model that is an expert in law, finance or medicine might perform at the Expert level in general conversation, but superhuman in its specific domain.
For example, current leading language models (e.g., ChatGPT, Bard, Llama 2, etc.) achieve ‘Competent’ level performance on some tasks (e.g., short essay writing, simple coding), but remain at ‘Emerging’ level performance on most tasks (e.g., mathematical ability, factual tasks). Therefore, they are considered to be Level 1 general AI (‘Emerging AGI’) until their performance levels improve across a wider range of tasks.
The highest level, ASI (Artificial Superintelligence), means that it outperforms humans in every task that humans can do. For example, AlphaFold is classified as a 'Superhuman Narrow AI' because it performs a single task (predicting the 3D structure of a protein from its amino acid sequence) at a level that surpasses the world's best scientists.
This taxonomy requires specific benchmarks for the depth and breadth of tasks that AGI can handle, and emphasizes that actual performance when deployed may not match actual performance. For example, user interface limitations may result in deployed performance being lower than theoretically possible performance.
In addition, the order in which AI acquires strong skills in specific cognitive domains can have serious implications for AI safety. For example, acquiring strong knowledge of chemical engineering before acquiring strong ethical reasoning skills can be a dangerous combination. Progress between levels of performance and generality can be nonlinear, and the ability to learn new skills can accelerate progress to the next level in particular.
Subscribe to 'haebom'
📚 Welcome to Haebom's archives. --- I post articles related to IT 💻, economy 💰, and humanities 🎭. If you are curious about my thoughts, perspectives or interests, please subscribe. Would you like to be notified when new articles are posted? 🔔 Yes, that means subscribe. haebom@kakao.com
