Sign In

Test of Time: Rethinking Temporal Signal of Benchmark Contamination

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
Empty

์ €์ž

Terry Jingchen Zhang, Gopal Dev, Ning Wang, Max Obreiter, Punya Syon Pandey, Keenan Samway, Wenyuan Jiang, Yinya Huang, Bernhard Scholkopf, Mrinmaya Sachan, Zhijing Jin

๐Ÿ’ก ๊ฐœ์š”

๋ณธ ๋…ผ๋ฌธ์€ ๋ฒค์น˜๋งˆํฌ ์˜ค์—ผ์˜ ์ฃผ์š” ์ง€ํ‘œ๋กœ ๊ฐ„์ฃผ๋˜์–ด ์˜จ '์‚ฌํ›„ ์ปท์˜คํ”„ ์„ฑ๋Šฅ ์ €ํ•˜' ํ˜„์ƒ์ด ์งˆ๋ฌธ ์ƒ์„ฑ ๋ฐฉ์‹์— ๋”ฐ๋ผ ํฌ๊ฒŒ ๋‹ฌ๋ผ์งˆ ์ˆ˜ ์žˆ์Œ์„ ๋น„ํŒ์ ์œผ๋กœ ๊ฒ€ํ† ํ•ฉ๋‹ˆ๋‹ค. ํŠนํžˆ, LLM์ด ์ƒ์„ฑํ•œ ์งˆ๋ฌธ๊ณผ ์ง์ ‘ ๊ฒ€์ƒ‰๋œ ๋นˆ์นธ ์ฑ„์šฐ๊ธฐ ์งˆ๋ฌธ ๊ฐ„์˜ ๋šœ๋ ทํ•˜๊ฒŒ ๋‹ค๋ฅธ ์‹œ๊ฐ„์  ํŒจํ„ด์„ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ๋ฐœ๊ฒฌ์„ ํ†ตํ•ด ๊ธฐ์กด ๋ฒค์น˜๋งˆํฌ์˜ ์‹œ๊ฐ„์  ์˜ค์—ผ ์‹ ํ˜ธ์˜ ๋ฏผ๊ฐ์„ฑ์„ ์กฐ๋ช…ํ•˜๊ณ , ๋”์šฑ ์‹ ๋ขฐํ•  ์ˆ˜ ์žˆ๋Š” AI ํ‰๊ฐ€๋ฅผ ์œ„ํ•œ ๊ฒฌ๊ณ ํ•œ ์˜ค์—ผ ํƒ์ง€ ๋ฐฉ๋ฒ•์˜ ํ•„์š”์„ฑ์„ ๊ฐ•์กฐํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
๋ฒค์น˜๋งˆํฌ ์˜ค์—ผ ํƒ์ง€์— ์‚ฌ์šฉ๋˜๋Š” '์‚ฌํ›„ ์ปท์˜คํ”„ ์„ฑ๋Šฅ ์ €ํ•˜'๋ผ๋Š” ์‹œ๊ฐ„์  ์‹ ํ˜ธ๊ฐ€ ์งˆ๋ฌธ ์ƒ์„ฑ ๋ฐฉ์‹์— ๋”ฐ๋ผ ๋งค์šฐ ๋ฏผ๊ฐํ•˜๊ฒŒ ๋ณ€ํ™”ํ•˜๋ฏ€๋กœ, ์ด๋ฅผ ๋งน์‹ ํ•ด์„œ๋Š” ์•ˆ ๋ฉ๋‹ˆ๋‹ค.
โ€ข
LLM์„ ํ™œ์šฉํ•œ ์งˆ๋ฌธ ์ƒ์„ฑ ๋ฐฉ์‹์ด ์˜๋„์น˜ ์•Š์€ ์‹œ๊ฐ„์  ์˜ค์—ผ ์‹ ํ˜ธ๋ฅผ ์•ผ๊ธฐํ•  ์ˆ˜ ์žˆ์œผ๋ฉฐ, ๊ฐ„๋‹จํ•œ LLM ๋ณ€ํ™˜๋งŒ์œผ๋กœ๋„ ์ด๋Ÿฌํ•œ ์‹ ํ˜ธ๋ฅผ ์ œ๊ฑฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
โ€ข
๋ณธ ์—ฐ๊ตฌ๋Š” ์งˆ๋ฌธ ์ƒ์„ฑ ๋ฐฉ์‹์˜ ์ค‘์š”์„ฑ์„ ๊ฐ•์กฐํ•˜๋ฉฐ, ํ–ฅํ›„ AI ๋ฒค์น˜๋งˆํฌ ์„ค๊ณ„ ๋ฐ ์˜ค์—ผ ํƒ์ง€ ๋ฐฉ๋ฒ•๋ก  ๊ฐœ๋ฐœ์— ์ƒˆ๋กœ์šด ๊ด€์ ์„ ์ œ์‹œํ•ฉ๋‹ˆ๋‹ค.
โ€ข
์•„์ง LLM์ด ์ƒ์„ฑํ•œ ์งˆ๋ฌธ์ด ์‹œ๊ฐ„์  ํŒจํ„ด์— ์˜ํ–ฅ์„ ๋ฏธ์น˜๋Š” ๊ทผ๋ณธ์ ์ธ ๋ฉ”์ปค๋‹ˆ์ฆ˜์— ๋Œ€ํ•œ ์‹ฌ์ธต์ ์ธ ๋ถ„์„์ด ๋” ํ•„์š”ํ•˜๋ฉฐ, ๋‹ค์–‘ํ•œ ๋ฒค์น˜๋งˆํฌ์™€ ๋ชจ๋ธ์— ๋Œ€ํ•œ ์ผ๋ฐ˜ํ™” ๊ฐ€๋Šฅ์„ฑ์„ ์ถ”๊ฐ€์ ์œผ๋กœ ๊ฒ€์ฆํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
๐Ÿ‘
Made with Slashpage