Sign In

Omni-C: Compressing Heterogeneous Modalities into a Single Dense Encoder

Created by
  • Haebom
Category
Empty

์ €์ž

Kin Wai Lau, Yasar Abbas Ur Rehman, Lai-Man Po, Pedro Porto Buarque de Gusmao

๐Ÿ’ก ๊ฐœ์š”

๊ธฐ์กด ๋‹ค์ค‘ ๋ชจ๋‹ฌ ์‹œ์Šคํ…œ์€ ๋ชจ๋‹ฌ๋ฆฌํ‹ฐ๊ฐ€ ์ถ”๊ฐ€๋ ์ˆ˜๋ก ๋ณต์žก์„ฑ๊ณผ ์—ฐ์‚ฐ๋Ÿ‰์ด ์„ ํ˜•์ ์œผ๋กœ ์ฆ๊ฐ€ํ•˜๋Š” ๋ฌธ์ œ๋ฅผ ์•ˆ๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๋ณธ ๋…ผ๋ฌธ์€ ์ด๋Ÿฌํ•œ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด ์ด๋ฏธ์ง€, ์˜ค๋””์˜ค, ํ…์ŠคํŠธ ๋“ฑ ์ด์งˆ์ ์ธ ๋ชจ๋‹ฌ๋ฆฌํ‹ฐ๋ฅผ ๋‹จ์ผ ๋ฐ€์ง‘ ํŠธ๋žœ์Šคํฌ๋จธ ์ธ์ฝ”๋”์— ์••์ถ•ํ•˜๋Š” Omni-C๋ฅผ ์ œ์•ˆํ•ฉ๋‹ˆ๋‹ค. Omni-C๋Š” ๋Œ€๊ทœ๋ชจ ์ •๋ ฌ๋˜์ง€ ์•Š์€ ๋ฐ์ดํ„ฐ๋ฅผ ํ™œ์šฉํ•œ ๋‹จ์ผ ๋ชจ๋‹ฌ ๋Œ€์กฐ ํ•™์Šต์„ ํ†ตํ•ด ๊ฒฝ์Ÿ๋ ฅ ์žˆ๋Š” ๊ณต์œ  ํ‘œํ˜„์„ ํ•™์Šตํ•˜๋ฉฐ, MoE๋‚˜ ๋ผ์šฐํŒ… ์—†์ด ํšจ์œจ์ ์ธ ๋ฐฐํฌ๋ฅผ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
์ด์งˆ์ ์ธ ๋‹ค์ค‘ ๋ชจ๋‹ฌ๋ฆฌํ‹ฐ๋ฅผ ๋‹จ์ผ ๋ฐ€์ง‘ ์ธ์ฝ”๋”๋กœ ํšจ์œจ์ ์œผ๋กœ ์••์ถ•ํ•˜์—ฌ ๋ชจ๋ธ ๋ณต์žก์„ฑ๊ณผ ์—ฐ์‚ฐ๋Ÿ‰ ์ฆ๊ฐ€ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•ฉ๋‹ˆ๋‹ค.
โ€ข
MoE, ์Œ๋ณ„ ์ง€๋„ ํ•™์Šต, ๋ผ์šฐํŒ… ์—†์ด๋„ ๊ฒฝ์Ÿ๋ ฅ ์žˆ๋Š” ๊ณต์œ  ํ‘œํ˜„ ํ•™์Šต์ด ๊ฐ€๋Šฅํ•˜๋ฉฐ, ๋ฉ”๋ชจ๋ฆฌ ์ œ์•ฝ์ ์ธ ํ™˜๊ฒฝ์—์„œ์˜ ํšจ์œจ์ ์ธ ๋ฐฐํฌ์— ๊ฐ•์ ์„ ๋ณด์ž…๋‹ˆ๋‹ค.
โ€ข
์˜ค๋””์˜ค ๋ฐ ํ…์ŠคํŠธ์— ๋Œ€ํ•œ ์ œ๋กœ์ƒท ์„ฑ๋Šฅ์—์„œ ์•ฝ๊ฐ„์˜ ์ €ํ•˜๊ฐ€ ๊ด€์ฐฐ๋˜๋‚˜, ์„ ํ˜• ํƒ์ƒ‰์ด๋‚˜ ํŒŒ๋ผ๋ฏธํ„ฐ ํšจ์œจ์  ๋ฏธ์„ธ ์กฐ์ •์„ ํ†ตํ•ด ์ƒ๋‹น ๋ถ€๋ถ„ ํšŒ๋ณต ๊ฐ€๋Šฅํ•ฉ๋‹ˆ๋‹ค.
โ€ข
๋Œ€๊ทœ๋ชจ ์ •๋ ฌ๋˜์ง€ ์•Š์€ ๋ฐ์ดํ„ฐ์…‹๊ณผ ๋‹ค์–‘ํ•œ ๋ฒค์น˜๋งˆํฌ์— ๋Œ€ํ•œ ์‹คํ—˜์„ ํ†ตํ•ด ์ œ์•ˆ๋œ ๋ฐฉ๋ฒ•๋ก ์˜ ํšจ๊ณผ์„ฑ์„ ์ž…์ฆํ–ˆ์Šต๋‹ˆ๋‹ค.
๐Ÿ‘