Sign In

MultiVer: Zero-Shot Multi-Agent Vulnerability Detection

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
Empty

์ €์ž

Shreshth Rajan

๐Ÿ’ก ๊ฐœ์š”

๋ณธ ๋…ผ๋ฌธ์€ ๋ฏธ์„ธ ์กฐ์ • ์—†์ด ์ œ๋กœ์ƒท(zero-shot) ๋ฐฉ์‹์œผ๋กœ ์ทจ์•ฝ์ ์„ ํƒ์ง€ํ•˜๋Š” ๋ฉ€ํ‹ฐ ์—์ด์ „ํŠธ ์‹œ์Šคํ…œ์ธ MultiVer์„ ์ œ์•ˆํ•œ๋‹ค. ๋„ค ๊ฐ€์ง€ ์ „๋ฌธ ์—์ด์ „ํŠธ(๋ณด์•ˆ, ์ •ํ™•์„ฑ, ์„ฑ๋Šฅ, ์Šคํƒ€์ผ)์˜ ์•™์ƒ๋ธ”๊ณผ ํ†ตํ•ฉ ํˆฌํ‘œ ๋ฐฉ์‹์„ ํ†ตํ•ด PyVul ๋ฒค์น˜๋งˆํฌ์—์„œ ๋ฏธ์„ธ ์กฐ์ •๋œ GPT-3.5๋ฅผ ๋Šฅ๊ฐ€ํ•˜๋Š” 82.7%์˜ ๋ฆฌ์ฝœ(recall) ์„ฑ๋Šฅ์„ ๋‹ฌ์„ฑํ–ˆ์œผ๋ฉฐ, SecurityEval์—์„œ๋„ 91.7%์˜ ํƒ์ง€์œจ์„ ๊ธฐ๋กํ–ˆ๋‹ค. ์ด๋Š” ๋ณด์•ˆ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์—์„œ ๋” ์ค‘์š”ํ•œ ๋ฆฌ์ฝœ ์„ฑ๋Šฅ์„ ์ œ๋กœ์ƒท ๋ฉ€ํ‹ฐ ์—์ด์ „ํŠธ ์‹œ์Šคํ…œ์œผ๋กœ ํ–ฅ์ƒ์‹œํ‚ฌ ์ˆ˜ ์žˆ์Œ์„ ๋ณด์—ฌ์ค€๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
๋ณด์•ˆ ํƒ์ง€์—์„œ ๋ฆฌ์ฝœ(๋ฏผ๊ฐ๋„) ์„ฑ๋Šฅ์ด ์ •๋ฐ€๋„(์ •ํ™•์„ฑ)๋ณด๋‹ค ์ค‘์š”ํ•  ๊ฒฝ์šฐ, ์ œ๋กœ์ƒท ๋ฉ€ํ‹ฐ ์—์ด์ „ํŠธ ์‹œ์Šคํ…œ์ด ๋ฏธ์„ธ ์กฐ์ •๋œ ๋ชจ๋ธ์„ ๋Šฅ๊ฐ€ํ•  ์ˆ˜ ์žˆ๋‹ค.
โ€ข
๋ฉ€ํ‹ฐ ์—์ด์ „ํŠธ ์•™์ƒ๋ธ” ๊ตฌ์กฐ๋Š” ๋‹จ์ผ ์—์ด์ „ํŠธ ๋Œ€๋น„ ๋ฆฌ์ฝœ ์„ฑ๋Šฅ์„ ํฌ๊ฒŒ ํ–ฅ์ƒ์‹œํ‚ค๋Š” ๋ฐ ๊ธฐ์—ฌํ•œ๋‹ค.
โ€ข
์ œ์•ˆ๋œ ์‹œ์Šคํ…œ์€ ๋ฆฌ์ฝœ ์„ฑ๋Šฅ์„ ๋†’์ด๋Š” ๋Œ€์‹  ์ •๋ฐ€๋„๊ฐ€ ๋‚ฎ์•„์ง€๋Š” ์ ˆ์ถฉ(trade-off)์ด ๋ฐœ์ƒํ•˜๋ฉฐ, ์ด๋Š” F1 ์ ์ˆ˜์— ์˜ํ–ฅ์„ ๋ฏธ์นœ๋‹ค.
๐Ÿ‘
Made with Slashpage