On April 19, 2024, LLaMA3 developed by MetaAI was released. There was also Llama and Llama2, so what's the fuss about? The release of this model has a greater meaning than you might think. It's also amazing that GPT-4 is still maintaining the top performance. (What if GPT-5 comes out soon...?)
Performance
First of all, in terms of performance, the LLaMA3 model is available in various sizes, including 8B, 70B, and the upcoming 400B. These models boast top-level performance in areas such as conversation, inference, code generation, and instruction following. This is showing meaningful results not only in the benchmarks they have released, but also in the actual chatbot arena.
Personally, when I actually used it, it clearly surpassed GPT-3.5 in English, and showed performance similar to GPT-4 in most tasks. Considering that 400B has not been released yet, I am confident that an upward performance standardization in language models will occur soon.
They say they support responsible AI use through various trust and security tools such as Llama Guard 2, Code Shield, and CyberSec Eval 2, but I don't know much about this area, so I think they've figured it out. The important thing is the license.
What’s even more surprising is that Meta has still released LLaMA3 as open source. Now, developers around the world can freely use this technology, improve it, and explore new ideas. This will not only popularize AI research and development, enable more innovation, but also increase the transparency and reliability of AI technology. In fact, considering that LLaMA2 came out and fine-tuned models based on it were released, we can expect to see even more diverse models this time.
The improved performance of LLaMA3 has further expanded the possibilities of utilizing AI in various fields such as education, medicine, and finance. For example, the sophisticated natural language processing ability can be of great help in analyzing legal documents, interpreting complex medical information, and predicting financial markets. It seems that AI technology will now be more deeply involved in solving problems in our daily lives. When releasing it this time, unlike LLaMA2, hurdles such as waiting in line were completely eliminated, and APIs and other things were opened so that it could be used quickly. Seeing this, I wondered if they were really trying to do AGI meta.
Of course, with the announcement of the LLaMA3 license, commercial use has become somewhat more difficult. However, from the research perspective, I am grateful.
Usability
MetaAI is based on LLaMA3 technology. With the integrated search function in the MetaAI app, you can find real-time information and complete desired tasks without having to switch between apps. The 'Imagine' function not only creates images from text in real time, but also allows you to create simple GIFs. It is said that it will be sequentially applied to Facebook, Instagram, WhatsApp, etc. in the future.
This Rama3 will provide many opportunities to jump into language model development and will likely be a starting point for improving performance. In addition, it is impressive that a lot of effort was put into quality control while handling a huge dataset of 15 trillion tokens. It must have been a really important process to extract only high-quality data through heuristic filters, NSFW filters, and semantic-based deduplication techniques.
In particular, it is noteworthy that the ratio of multilingual content has been significantly increased compared to the previous model. It must not have been easy to secure high-quality multilingual data with the goal of supporting 30 languages, but it is a truly remarkable achievement to be able to properly grasp the nuances of each language.
In terms of learning methodology, the Single Epoch approach was adopted to increase efficiency, and the data composition was optimized by increasing the proportion of code data. In addition, various technical elements such as the 128K tokenizer, GQA technology, and 8K Context Length were combined to further improve language processing performance, and the effort and passion that Meta put into developing LLaMA3 was fully felt here.
📚 Welcome to Haebom's archives. --- I post articles related to IT 💻, economy 💰, and humanities 🎭. If you are curious about my thoughts, perspectives or interests, please subscribe. Would you like to be notified when new articles are posted? 🔔 Yes, that means subscribe. haebom@kakao.com
