Sign In

Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
Empty

์ €์ž

Jaehyeok Lee, Xiaoyuan Yi, Jing Yao, Hyunjin Hwang, Roy Ka-Wei Lee, Xing Xie, JinYeong Bak

๐Ÿ’ก ๊ฐœ์š”

์ด ๋…ผ๋ฌธ์€ ๊ธ€๋กœ๋ฒŒ ๋ฐฐํฌ๋˜๋Š” LLM์˜ ๋ฌธํ™”์  ๊ฐ€์น˜ ์ •๋ ฌ์˜ ์ค‘์š”์„ฑ์„ ๊ฐ•์กฐํ•˜๋ฉฐ, ๊ธฐ์กด ํ‰๊ฐ€ ๋ฐฉ์‹์˜ ํ•œ๊ณ„๋ฅผ ๊ทน๋ณตํ•˜๊ธฐ ์œ„ํ•ด 'DOVE'๋ผ๋Š” ์ƒˆ๋กœ์šด ๋ถ„ํฌ ๊ธฐ๋ฐ˜ ํ‰๊ฐ€ ํ”„๋ ˆ์ž„์›Œํฌ๋ฅผ ์ œ์•ˆํ•ฉ๋‹ˆ๋‹ค. DOVE๋Š” ์ธ๊ฐ„ ์ž‘์„ฑ ํ…์ŠคํŠธ์™€ LLM ์ƒ์„ฑ ํ…์ŠคํŠธ์˜ ๋ถ„ํฌ๋ฅผ ์ง์ ‘ ๋น„๊ตํ•˜๊ณ , ์••์ถ•๋œ ๊ฐ€์น˜ ์ฝ”๋“œ๋ถ๊ณผ ์ตœ์  ์ˆ˜์†ก์„ ํ™œ์šฉํ•˜์—ฌ ๋ฌธํ™” ๋‚ด ๋‹ค์–‘์„ฑ๊ณผ ํ•˜์œ„ ๊ทธ๋ฃน์˜ ์ฐจ์ด๋ฅผ ํฌ์ฐฉํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
LLM์˜ ๋ฌธํ™”์  ๊ฐ€์น˜ ์ •๋ ฌ์„ ์‹ค์ œ ์‚ฌ์šฉ์ž ๊ฒฝํ—˜์— ๋” ๊ฐ€๊น๊ฒŒ ํ‰๊ฐ€ํ•  ์ˆ˜ ์žˆ๋Š” ์ƒˆ๋กœ์šด ๋ฐฉ๋ฒ•๋ก ์„ ์ œ์‹œํ•ฉ๋‹ˆ๋‹ค.
โ€ข
๊ธฐ์กด์˜ ์„ ํƒํ˜• ์งˆ๋ฌธ ๋ฐฉ์‹์—์„œ ๋ฒ—์–ด๋‚˜, ํ…์ŠคํŠธ ๋ถ„ํฌ ๋น„๊ต๋ฅผ ํ†ตํ•ด LLM์˜ ์ง„์ •ํ•œ ๊ฐ€์น˜ ์ง€ํ–ฅ์„ฑ์„ ํ‰๊ฐ€ํ•˜๋Š” ๋ฐ ํšจ๊ณผ์ ์ž…๋‹ˆ๋‹ค.
โ€ข
10,000๊ฐœ์˜ ๋ฌธ์„œ๋กœ๋ถ€ํ„ฐ ์••์ถ•๋œ ๊ฐ€์น˜ ์ฝ”๋“œ๋ถ์„ ๊ตฌ์ถ•ํ•˜๊ณ , ๋ถˆ๊ท ํ˜• ์ตœ์  ์ˆ˜์†ก์„ ํ†ตํ•ด ๋ฌธํ™”์  ํ•˜์œ„ ๊ทธ๋ฃน ๋‹ค์–‘์„ฑ์„ ์ธก์ •ํ•˜๋Š” ๋ฐฉ๋ฒ•๋ก ์€ LLM์˜ ๋ฌธํ™”์  ํŽธํ–ฅ์„ ๋ถ„์„ํ•˜๋Š” ๋ฐ ์ค‘์š”ํ•œ ๊ธฐ์—ฌ๋ฅผ ํ•ฉ๋‹ˆ๋‹ค.
โ€ข
์ œ์•ˆ๋œ ๋ฐฉ๋ฒ•๋ก ์˜ ๊ณ„์‚ฐ ๋ณต์žก์„ฑ ๋ฐ ๋Œ€๊ทœ๋ชจ ๋ฌธํ™” ๋ฐ์ดํ„ฐ์…‹ ๊ตฌ์ถ•์˜ ์–ด๋ ค์›€์ด ํ–ฅํ›„ ์—ฐ๊ตฌ ๊ณผ์ œ๋กœ ๋‚จ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
๐Ÿ‘
Made with Slashpage