Sign In

DeEscalWild: A Real-World Benchmark for Automated De-Escalation Training with SLMs

Created by
  • Haebom
Category
Empty

์ €์ž

Md Hasebul Hasan, Krity Haque Charu, Eshwara Prasad Sridhar, Shuchisnigdha Deb, Mohammad A. Islam

๐Ÿ’ก ๊ฐœ์š”

๋ณธ ๋…ผ๋ฌธ์€ ๊ฒฝ์ฐฐ๊ด€์˜ ํšจ๊ณผ์ ์ธ ๋น„์ƒ ์ƒํ™ฉ ํ•ด์ œ ํ›ˆ๋ จ์„ ์œ„ํ•œ ํ˜„์‹ค์ ์ธ ๋ฒค์น˜๋งˆํฌ ๋ฐ์ดํ„ฐ์…‹์ธ DeEscalwild๋ฅผ ์ œ์•ˆํ•ฉ๋‹ˆ๋‹ค. LLM์˜ ๋†’์€ ์—ฐ์‚ฐ๋Ÿ‰์„ ๊ทน๋ณตํ•˜๊ธฐ ์œ„ํ•ด SLM์„ ํ™œ์šฉํ•˜๊ณ ์ž ํ–ˆ์œผ๋‚˜, ๊ด€๋ จ ๋ฐ์ดํ„ฐ ๋ถ€์กฑ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด ์‹ค์ œ ๊ฒฝ์ฐฐ-์‹œ๋ฏผ ์ƒํ˜ธ์ž‘์šฉ ์˜์ƒ์—์„œ ์ถ”์ถœ ๋ฐ ์ •์ œ๋œ 1,500๊ฐœ์˜ ์‹œ๋‚˜๋ฆฌ์˜ค๋กœ ๊ตฌ์„ฑ๋œ ๋Œ€๊ทœ๋ชจ ๋ฐ์ดํ„ฐ์…‹์„ ๊ตฌ์ถ•ํ–ˆ์Šต๋‹ˆ๋‹ค. ์ด ๋ฐ์ดํ„ฐ์…‹์œผ๋กœ ํ•™์Šต๋œ SLM์€ ๋‹ค์–‘ํ•œ ํ‰๊ฐ€ ์ง€ํ‘œ์—์„œ ๋›ฐ์–ด๋‚œ ์„ฑ๋Šฅ์„ ๋ณด์—ฌ, ๊ฒฝ๋Ÿ‰ํ™”๋œ ์—ฃ์ง€ ๋””๋ฐ”์ด์Šค์—์„œ๋„ ํšจ๊ณผ์ ์ธ ํ›ˆ๋ จ ์‹œ์Šคํ…œ ๊ตฌ์ถ• ๊ฐ€๋Šฅ์„ฑ์„ ์ œ์‹œํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
ํ˜„์‹ค ๊ธฐ๋ฐ˜ ํ›ˆ๋ จ ๋ฐ์ดํ„ฐ์…‹ ๊ตฌ์ถ•: ์‹ค์ œ ์‚ฌ๊ฑด์—์„œ ์ถ”์ถœ๋œ ๊ณ ํ’ˆ์งˆ ๋ฐ์ดํ„ฐ์…‹์€ ๊ณต๊ณต ์•ˆ์ „ ๋ถ„์•ผ์—์„œ AI ๊ธฐ๋ฐ˜ ํ›ˆ๋ จ ์‹œ์Šคํ…œ ๊ฐœ๋ฐœ์˜ ์ค‘์š”ํ•œ ๊ธฐ๋ฐ˜์ด ๋  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
โ€ข
SLM์˜ ์ž ์žฌ๋ ฅ ์ž…์ฆ: ์—ฐ์‚ฐ๋Ÿ‰์ด ์ ์€ SLM๋„ ํŠน์ • ๋„๋ฉ”์ธ์— ์ตœ์ ํ™”๋œ ๋ฐ์ดํ„ฐ๋ฅผ ํ†ตํ•ด ๋ฒ”์šฉ LLM์— ๋ฒ„๊ธˆ๊ฐ€๊ฑฐ๋‚˜ ๋Šฅ๊ฐ€ํ•˜๋Š” ์„ฑ๋Šฅ์„ ๋‚ผ ์ˆ˜ ์žˆ์Œ์„ ๋ณด์—ฌ์ฃผ์—ˆ์Šต๋‹ˆ๋‹ค.
โ€ข
์—ฃ์ง€ ์ปดํ“จํŒ…์—์„œ์˜ ํ›ˆ๋ จ ์‹œ์Šคํ…œ ๊ฐ€๋Šฅ์„ฑ: ๋‚ฎ์€ ์ง€์—ฐ ์‹œ๊ฐ„๊ณผ ๊ฐœ์ธ ์ •๋ณด ๋ณดํ˜ธ๋ฅผ ๊ณ ๋ คํ•œ ์—ฃ์ง€ ๋””๋ฐ”์ด์Šค์—์„œ์˜ ์‹ค์‹œ๊ฐ„ ํ›ˆ๋ จ ์‹œ์Šคํ…œ ๊ตฌ์ถ• ๊ฐ€๋Šฅ์„ฑ์„ ์—ด์—ˆ์Šต๋‹ˆ๋‹ค.
โ€ข
๋ฐ์ดํ„ฐ์˜ ํŽธํ–ฅ์„ฑ ๋ฐ ์ผ๋ฐ˜ํ™”: 'in-the-wild' ๋ฐ์ดํ„ฐ์˜ ํŠน์„ฑ์ƒ ํŠน์ • ์ƒํ™ฉ์ด๋‚˜ ์ƒํ˜ธ์ž‘์šฉ ํŒจํ„ด์— ํŽธํ–ฅ๋  ๊ฐ€๋Šฅ์„ฑ์ด ์žˆ์œผ๋ฉฐ, ๋‹ค์–‘ํ•œ ์ธ์ข…, ๋ฐฐ๊ฒฝ, ์ƒํ™ฉ์— ๋Œ€ํ•œ ์ผ๋ฐ˜ํ™” ์„ฑ๋Šฅ์€ ์ถ”๊ฐ€ ์—ฐ๊ตฌ๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.
๐Ÿ‘
Made with Slashpage