๋กœ๊ทธ์ธ

ANGOFA: Leveraging OFA Embedding Initialization and Synthetic Data for Angolan Language Model

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
๋น„์–ด ์žˆ์Œ

์ €์ž

Osvaldo Luamba Quinjica, David Ifeoluwa Adelani

๐Ÿ’ก ๊ฐœ์š”

๋ณธ ๋…ผ๋ฌธ์€ ์ž์›์ด ๋งค์šฐ ๋ถ€์กฑํ•œ ์•™๊ณจ๋ผ ์–ธ์–ด๋ฅผ ์œ„ํ•œ 4๊ฐœ์˜ ๋งž์ถคํ˜• ์‚ฌ์ „ ํ•™์Šต ์–ธ์–ด ๋ชจ๋ธ(PLM)์„ ๊ฐœ๋ฐœํ•˜๊ณ , OFA ์ž„๋ฒ ๋”ฉ ์ดˆ๊ธฐํ™”์™€ ํ•ฉ์„ฑ ๋ฐ์ดํ„ฐ๋ฅผ ํ™œ์šฉํ•œ ๋‹ค๊ตญ์–ด ์ ์‘ ๋ฏธ์„ธ ์กฐ์ •(MAFT) ์ ‘๊ทผ ๋ฐฉ์‹์„ ์ œ์•ˆํ•ฉ๋‹ˆ๋‹ค. ์ด๋ฅผ ํ†ตํ•ด ๊ธฐ์กด ์ตœ์‹  ๊ธฐ์ˆ ์ธ AfroXLMR-base ๋ฐ OFA ๋ชจ๋ธ์˜ ์„ฑ๋Šฅ์„ ๊ฐ๊ฐ 12.3%์™€ 3.8% ํ–ฅ์ƒ์‹œ์ผฐ์Šต๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
์ž์›์ด ๋ถ€์กฑํ•œ ์–ธ์–ด์— ๋Œ€ํ•œ PLM ๊ฐœ๋ฐœ์˜ ์ค‘์š”์„ฑ์„ ๊ฐ•์กฐํ•˜๋ฉฐ, MAFT์™€ OFA ์ž„๋ฒ ๋”ฉ ์ดˆ๊ธฐํ™”๊ฐ€ ํšจ๊ณผ์ ์ธ ํ•ด๊ฒฐ์ฑ…์ด ๋  ์ˆ˜ ์žˆ์Œ์„ ์ œ์‹œํ•ฉ๋‹ˆ๋‹ค.
โ€ข
ํ•ฉ์„ฑ ๋ฐ์ดํ„ฐ ์ƒ์„ฑ์„ ํ†ตํ•ด ๋ถ€์กฑํ•œ ์–ธ์–ด ๋ฐ์ดํ„ฐ๋ฅผ ๋ณด์™„ํ•˜๋Š” ๋ฐฉ์•ˆ์˜ ์œ ํšจ์„ฑ์„ ์ž…์ฆํ•ฉ๋‹ˆ๋‹ค.
โ€ข
์ œ์‹œ๋œ ๋ฐฉ๋ฒ•๋ก ์˜ ์ผ๋ฐ˜ํ™” ๊ฐ€๋Šฅ์„ฑ๊ณผ ์‹ค์ œ ์ ์šฉ์„ ์œ„ํ•œ ์ถ”๊ฐ€์ ์ธ ์–ธ์–ด ๋ฐ ๋ฐ์ดํ„ฐ์…‹ ํ™•์žฅ ์—ฐ๊ตฌ๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.
๐Ÿ‘
Slashpage๋กœ ์ œ์ž‘๋จ