Sign In

CalBench: Evaluating Coordination-Privacy Trade-offs in Multi-Agent LLMs

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
Empty

์ €์ž

Chelsea Zou, Yiheng Yao, Selena She, Noah Goodman, Robert D. Hawkins

๐Ÿ’ก ๊ฐœ์š”

๊ฐœ์ธ AI ๋น„์„œ๊ฐ€ ์‚ฌ์šฉ์ž๋ฅผ ๋Œ€ํ‘œํ•˜์—ฌ ๋‹ค๋ฅธ ๋น„์„œ์™€ ์ผ์ •์„ ์กฐ์œจํ•ด์•ผ ํ•˜๋Š” ์ƒํ™ฉ์—์„œ, ์–ด๋–ค ์ •๋ณด๋ฅผ ๊ณต๊ฐœํ•  ๊ฒƒ์ธ์ง€์— ๋Œ€ํ•œ ์‹ ๋ขฐ ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค. ๋ณธ ์—ฐ๊ตฌ๋Š” ๋‹ค์ค‘ ์—์ด์ „ํŠธ์˜ ๊ฐœ์ธ ์ •๋ณด ๋ณดํ˜ธ๋ฅผ ๊ณ ๋ คํ•œ ์บ˜๋ฆฐ๋” ์ผ์ • ๊ด€๋ฆฌ ์ž‘์—…์„ ์œ„ํ•œ ํ†ต์ œ๋œ ๋ฒค์น˜๋งˆํฌ์ธ CalBench๋ฅผ ์ œ์•ˆํ•ฉ๋‹ˆ๋‹ค. CalBench๋Š” ์–ธ์–ด ๊ธฐ๋ฐ˜ ํ˜‘์ƒ์„ ํ†ตํ•ด ์ค‘์•™ ์ง‘์ค‘์‹ ๊ณ„ํš ์—†์ด๋„ ๊ณผ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๊ณ , ๋‹ค์–‘ํ•œ AI ๋ชจ๋ธ์˜ ์„ฑ๋Šฅ์„ ํ‰๊ฐ€ํ•˜์—ฌ ํ˜‘์ƒ, ๋น„์šฉ, ํ†ต์‹  ํšจ์œจ์„ฑ, ๊ณต์ •์„ฑ ๋ฐ ๊ฐœ์ธ ์ •๋ณด ๋ณดํ˜ธ ๋ฌธ์ œ๋ฅผ ๋ถ„์„ํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
AI ๋น„์„œ๋“ค์ด ์‚ฌ์šฉ์ž๋ฅผ ๋Œ€ํ‘œํ•˜์—ฌ ์ผ์ •์„ ์กฐ์œจํ•  ๋•Œ, ๋‹จ์ˆœํžˆ ์ผ์ •์„ ์™„๋ฃŒํ•˜๋Š” ๊ฒƒ ์ด์ƒ์œผ๋กœ ๋น„์šฉ ํšจ์œจ์„ฑ, ํ†ต์‹ ๋Ÿ‰, ๊ณต์ •ํ•œ ์ฑ…์ž„ ๋ถ„๋‹ด, ๊ทธ๋ฆฌ๊ณ  ๊ฐœ์ธ ์ •๋ณด ๋ณดํ˜ธ ์‚ฌ์ด์˜ ๊ท ํ˜•์„ ์‹ ์ค‘ํ•˜๊ฒŒ ๊ณ ๋ คํ•ด์•ผ ํ•จ์„ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค.
โ€ข
๊ฐœ์ธ ์ •๋ณด ๋ณดํ˜ธ๋ฅผ ์œ„ํ•œ ์นจ๋ฌต์ด ์˜คํžˆ๋ ค ํŒ€์›์˜ ๋น„์šฉ ์ •๋ณด๋ฅผ ์–ป์ง€ ๋ชปํ•˜๊ฒŒ ํ•˜์—ฌ ๊ณต์ •ํ•œ ์ฑ…์ž„ ๋ถ„๋‹ด์„ ์ €ํ•ดํ•  ์ˆ˜ ์žˆ์Œ์„ ์ง€์ ํ•ฉ๋‹ˆ๋‹ค.
โ€ข
AI ๋น„์„œ๋“ค์ด ๋Œ€๊ทœ๋ชจ๋กœ ๋ฐฐํฌ๋˜๊ธฐ ์ „์— ์‚ฌ์šฉ์ž๋ฅผ ๋Œ€์‹ ํ•˜์—ฌ ์„ฑ๊ณต์ ์œผ๋กœ ํ˜‘์—…ํ•  ์ˆ˜ ์žˆ๋Š”์ง€ ์—ฐ๊ตฌํ•˜๊ธฐ ์œ„ํ•œ ์žฌํ˜„ ๊ฐ€๋Šฅํ•œ ํ…Œ์ŠคํŠธ๋ฒ ๋“œ๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.
โ€ข
CalBench๋Š” ์–ธ์–ด ๊ธฐ๋ฐ˜ ํ˜‘์ƒ์„ ํ†ตํ•ด ๋ณต์žกํ•œ ๋‹ค์ค‘ ์—์ด์ „ํŠธ ์ž‘์—…์—์„œ ๋ฐœ์ƒํ•˜๋Š” ์กฐ์ •-๊ฐœ์ธ ์ •๋ณด ๋ณดํ˜ธ ํŠธ๋ ˆ์ด๋“œ์˜คํ”„๋ฅผ ํƒ์ƒ‰ํ•˜๋Š” ๋ฐ ์œ ์šฉํ•˜์ง€๋งŒ, ์‹ค์ œ ์‹œ๋‚˜๋ฆฌ์˜ค์˜ ๋ชจ๋“  ๋ณต์žก์„ฑ์„ ๋ฐ˜์˜ํ•˜๊ธฐ์—๋Š” ํ•œ๊ณ„๊ฐ€ ์žˆ์„ ์ˆ˜ ์žˆ์œผ๋ฉฐ, ๋ณด๋‹ค ์ •๊ตํ•œ ๊ฐœ์ธ ์ •๋ณด ๋ณดํ˜ธ ๋ฉ”์ปค๋‹ˆ์ฆ˜ ๋ฐ ํ˜‘์ƒ ์ „๋žต์— ๋Œ€ํ•œ ์ถ”๊ฐ€ ์—ฐ๊ตฌ๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.
๐Ÿ‘
Made with Slashpage