Sign In

TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agents

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
Empty

์ €์ž

Zhiqiang Liu, Wenhui Dong, Yilang Tan, Yuwen Qu, Haochen Yin, Chenyang Si

๐Ÿ’ก ๊ฐœ์š”

์‹ค์ œ ์ „๋ฌธ๊ฐ€ ์ˆ˜์ค€์˜ ์›Œํฌํ”Œ๋กœ์šฐ์—์„œ ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ ์ž…๋ ฅ์„ ํ•ด์„ํ•˜๊ณ , ๋„๊ตฌ๋ฅผ ํ˜‘๋ ฅ์ ์œผ๋กœ ์‚ฌ์šฉํ•˜๋ฉฐ, ์ค‘๊ฐ„ ๊ฒฐ๊ณผ๋ฌผ์„ ๊ฒ€ํ† ํ•˜๊ณ , ์ตœ์ข… ๊ฒฐ๊ณผ๋ฌผ์„ ์ƒ์„ฑํ•˜๊ธฐ ์ „์— ํ–‰๋™์„ ์ˆ˜์ •ํ•˜๋Š” ๋Šฅ๋ ฅ์ด ์š”๊ตฌ๋˜๋Š” ๋„๊ตฌ ์‚ฌ์šฉ ์—์ด์ „ํŠธ์— ๋Œ€ํ•œ ํ‰๊ฐ€์˜ ํ•„์š”์„ฑ์ด ๋Œ€๋‘๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ๋ณธ ๋…ผ๋ฌธ์€ ์ด๋Ÿฌํ•œ ํ˜„์‹ค ์„ธ๊ณ„์˜ ์š”๊ตฌ์‚ฌํ•ญ๊ณผ ๊ธฐ์กด ๋ฒค์น˜๋งˆํฌ ๊ฐ„์˜ ๊ฒฉ์ฐจ๋ฅผ ํ•ด์†Œํ•˜๊ธฐ ์œ„ํ•ด, ์‹คํ–‰ ๊ฐ€๋Šฅํ•œ 100๊ฐœ์˜ ์ž‘์—…์„ ํฌํ•จํ•˜๋Š” MM-ToolBench๋ผ๋Š” ์ƒˆ๋กœ์šด ๋ฒค์น˜๋งˆํฌ์™€ ํ‰๊ฐ€ ๋„๊ตฌ๋ฅผ ์ œ์•ˆํ•ฉ๋‹ˆ๋‹ค. MM-ToolBench๋Š” ํ์‡„ ๋ฃจํ”„ ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ ๊ฒ€์ฆ์„ ํ†ตํ•ด ์—์ด์ „ํŠธ๊ฐ€ ๋„๊ตฌ๋ฅผ ์‹คํ–‰ํ•˜๊ณ , ์‚ฐ์ถœ๋ฌผ์„ ๊ฒ€์‚ฌํ•˜๋ฉฐ, ํ•„์š”์‹œ ์Šค์Šค๋กœ ์ˆ˜์ •ํ•˜๋„๋ก ์œ ๋„ํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
์‹ค์ œ ์›Œํฌํ”Œ๋กœ์šฐ์— ๋Œ€ํ•œ ์—์ด์ „ํŠธ ์„ฑ๋Šฅ์˜ ๊ฒฉ์ฐจ ์‹œ์‚ฌ: ํ˜„์žฌ ์ตœ์ฒจ๋‹จ ์—์ด์ „ํŠธ์กฐ์ฐจ๋„ ์‹ค์ œ์™€ ๊ฐ™์€ ๋ณต์žกํ•œ ๋„๊ตฌ ์‚ฌ์šฉ ์ž‘์—…์—์„œ ๋‚ฎ์€ ์„ฑ๊ณต๋ฅ ์„ ๋ณด์—ฌ, ์‹ค์ œ ํ™˜๊ฒฝ์— ์ ์šฉํ•˜๊ธฐ ์œ„ํ•œ ์ƒ๋‹นํ•œ ๋ฐœ์ „์ด ํ•„์š”ํ•จ์„ ์‹œ์‚ฌํ•ฉ๋‹ˆ๋‹ค.
โ€ข
ํ์‡„ ๋ฃจํ”„ ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ ๊ฒ€์ฆ์˜ ์ค‘์š”์„ฑ ๊ฐ•์กฐ: ์—์ด์ „ํŠธ๊ฐ€ ๋‹จ์ˆœํžˆ ๋„๊ตฌ๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์„ ๋„˜์–ด, ์ค‘๊ฐ„ ๊ฒฐ๊ณผ๋ฌผ์„ ์ดํ•ดํ•˜๊ณ  ์Šค์Šค๋กœ ์ˆ˜์ •ํ•˜๋Š” ๋Šฅ๋ ฅ์˜ ์ค‘์š”์„ฑ์„ ๋ถ€๊ฐํ•˜๋ฉฐ, ์ด๋Ÿฌํ•œ ํ‰๊ฐ€ ๋ฐฉ์‹์˜ ์œ ์šฉ์„ฑ์„ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค.
โ€ข
ํ‰๊ฐ€ ์‹œ์Šคํ…œ์˜ ํ™•์žฅ์„ฑ ๋ฐ ๊ฒ€์ฆ ๊ฐ€๋Šฅ์„ฑ: MCP ๊ธฐ๋ฐ˜ ์‹คํ–‰, ์ž‘์—…๋ณ„ ํ‰๊ฐ€์ž, ๋ฐ˜์ž๋™ํ™”๋œ ์‹œ๋‚˜๋ฆฌ์˜ค ๊ตฌ์ถ• ํŒŒ์ดํ”„๋ผ์ธ์„ ํ†ตํ•ด ํ‰๊ฐ€์˜ ํ™•์žฅ์„ฑ๊ณผ ๊ฒ€์ฆ ๊ฐ€๋Šฅ์„ฑ์„ ํ™•๋ณดํ•œ ์ ์€ ํ–ฅํ›„ ์—ฐ๊ตฌ์— ์ค‘์š”ํ•œ ๊ธฐ๋ฐ˜์ด ๋ฉ๋‹ˆ๋‹ค.
โ€ข
ํ•œ๊ณ„์ : 100๊ฐœ์˜ ์ž‘์—…๊ณผ 27๊ฐœ์˜ MCP ์„œ๋ฒ„๋กœ ๊ตฌ์„ฑ๋˜์–ด ์žˆ์ง€๋งŒ, ์‹ค์ œ ์ „๋ฌธ๊ฐ€ ์›Œํฌํ”Œ๋กœ์šฐ์˜ ๋ฌดํ•œํ•œ ๋ณต์žก์„ฑ์„ ์™„์ „ํžˆ ๋ฐ˜์˜ํ•˜๊ธฐ์—๋Š” ์—ฌ์ „ํžˆ ํ•œ๊ณ„๊ฐ€ ์žˆ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋˜ํ•œ, ๋ฐ˜์ž๋™ํ™”๋œ ํŒŒ์ดํ”„๋ผ์ธ์˜ ํšจ๊ณผ์„ฑ๊ณผ ์ง€์†์ ์ธ ์—…๋ฐ์ดํŠธ ๋ฐ ๋‹ค์–‘ํ™”๊ฐ€ ์ค‘์š”ํ•œ ๊ณผ์ œ๊ฐ€ ๋  ๊ฒƒ์ž…๋‹ˆ๋‹ค.
๐Ÿ‘
Made with Slashpage