Sign In

Aligning Forest and Trees in Images & Long Captions for Visually Grounded Understanding

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
Empty

์ €์ž

Byeongju Woo, Zilin Wang, Byeonghyun Pak, Sangwoo Mo, Stella X. Yu

๐Ÿ’ก ๊ฐœ์š”

๋ณธ ๋…ผ๋ฌธ์€ CLIP๊ณผ ๊ฐ™์€ ๊ธฐ์กด ๋น„์ „-์–ธ์–ด ๋ชจ๋ธ์ด ๊ธด ์บก์…˜์˜ ์„ธ๋ถ€ ์‚ฌํ•ญ์„ ๋†“์น˜๊ณ  ์ฃผ๋กœ ์ง€๋ฐฐ์ ์ธ ์žฅ๋ฉด ์ •๋ณด์—๋งŒ ์˜์กดํ•˜๋Š” ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด ์ œ์•ˆ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ์ด๋ฅผ ์œ„ํ•ด CAFT (Cross-domain Alignment of Forests and Trees)๋ผ๋Š” ์ƒˆ๋กœ์šด ๋น„์ „-์–ธ์–ด ๋ชจ๋ธ์„ ๊ฐœ๋ฐœํ•˜์˜€์œผ๋ฉฐ, ์ด๋Š” ์ด๋ฏธ์ง€์˜ ์ง€์—ญ์  ์˜๋ฏธ์™€ ์ „์—ญ์  ์ด๋ฏธ์ง€-ํ…์ŠคํŠธ ์ •๋ ฌ์„ ๊ณต๋™์œผ๋กœ ํ•™์Šตํ•˜์—ฌ ๋ถ€๋ถ„-์ „์ฒด ๊ตฌ์„ฑ์œผ๋กœ ์žฅ๋ฉด์„ ์ดํ•ดํ•ฉ๋‹ˆ๋‹ค. CAFT๋Š” 3000๋งŒ ๊ฐœ์˜ ์ด๋ฏธ์ง€-ํ…์ŠคํŠธ ์Œ์œผ๋กœ ํ•™์Šต๋˜์–ด 6๊ฐ€์ง€ ์žฅ๋ฌธ ํ…์ŠคํŠธ ๊ฒ€์ƒ‰ ๋ฒค์น˜๋งˆํฌ์—์„œ ์ตœ์ฒจ๋‹จ ์„ฑ๋Šฅ์„ ๋‹ฌ์„ฑํ–ˆ์Šต๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
์ด๋ฏธ์ง€ ๋‚ด ์ง€์—ญ์  ํ…์ŠคํŠธ ์˜๋ฏธ๋ฅผ ๋ช…์‹œ์ ์ธ ์˜์—ญ ๋‹จ์œ„ ์ง€๋„ ํ•™์Šต ์—†์ด๋„ ์„ฑ๊ณต์ ์œผ๋กœ ํฌ์ฐฉํ•˜๊ณ  ํ•™์Šตํ•  ์ˆ˜ ์žˆ์Œ์„ ๋ณด์—ฌ์ค๋‹ˆ๋‹ค.
โ€ข
๊ธด ์บก์…˜์˜ ๊ณ„์ธต์  ๊ตฌ์กฐ๋ฅผ ํ™œ์šฉํ•˜์—ฌ ์ด๋ฏธ์ง€์™€ ํ…์ŠคํŠธ ๊ฐ„์˜ ๋ถ€๋ถ„-์ „์ฒด ๊ด€๊ณ„๋ฅผ ํšจ๊ณผ์ ์œผ๋กœ ๋ชจ๋ธ๋งํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
โ€ข
ํ…์ŠคํŠธ ์ธ์ฝ”๋”์˜ ๊ณ„์ธต์  ๊ตฌ์กฐ์™€ ์ด๋ฏธ์ง€ ์ธ์ฝ”๋”์˜ ๊ณ„์ธต์  ๊ตฌ์กฐ๋ฅผ ์กฐํ™”๋กญ๊ฒŒ ๊ฒฐํ•ฉํ•˜๋Š” ๋ฐฉ๋ฒ•๋ก ์ด ํšจ๊ณผ์ ์ž„์„ ์ž…์ฆํ–ˆ์Šต๋‹ˆ๋‹ค.
โ€ข
๋…ผ๋ฌธ์—์„œ ๋ช…์‹œ์ ์œผ๋กœ ์–ธ๊ธ‰ํ•˜์ง€๋Š” ์•Š์•˜์ง€๋งŒ, ๋‹ค์–‘ํ•œ ์œ ํ˜•์˜ ์ด๋ฏธ์ง€ ๋ฐ ์บก์…˜ ๋ฐ์ดํ„ฐ์— ๋Œ€ํ•œ ์ผ๋ฐ˜ํ™” ์„ฑ๋Šฅ์ด๋‚˜ ๊ณ„์‚ฐ ๋ณต์žก์„ฑ์— ๋Œ€ํ•œ ์ถ”๊ฐ€์ ์ธ ๋ถ„์„์ด ํ•„์š”ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
๐Ÿ‘
Made with Slashpage