구글 PaLM 2 파라미터 및 데이터 규모 유출 – 해봄

Haebom

May 18, 20233y ago

Google PaLM2 scale leak – 340B parameter size

The PaLM 2 report had issues with token count, so we couldn't know the exact scaling. Based on Chinchilla, we could only guess that it was at least 175B & 3.7T, but although the token count is similar, the model turns out to be almost twice as large.

Google's newest A.I. model uses nearly five times more text data for training than its predecessor

In announcing its PaLM 2 large language model, Google neglected to say how much training data was used for its most advanced LLM.

cnbc.com

Subscribe to 'haebom'

📚 Welcome to Haebom's archives.
---
I post articles related to IT 💻, economy 💰, and humanities 🎭.
If you are curious about my thoughts, perspectives or interests, please subscribe.
haebom@kakao.com