Sign In

Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
Empty

์ €์ž

Mert Yildiz, Pietro Spadaccino, Alexey Rolich, Francesca Cuomo, Andrea Baiocchi

๐Ÿ’ก ๊ฐœ์š”

๋ณธ ๋…ผ๋ฌธ์€ GPU ๋ฉ”๋ชจ๋ฆฌ ์ œ์•ฝ ํ•˜์—์„œ ์—ฌ๋Ÿฌ LLM ๋ชจ๋ธ์„ ํšจ์œจ์ ์œผ๋กœ ์„œ๋น™ํ•˜๊ธฐ ์œ„ํ•œ ๋‹ค์ค‘ ๋ชจ๋ธ ์Šค์ผ€์ค„๋Ÿฌ ์„ค๊ณ„์˜ ์–ด๋ ค์›€์„ ๋‹ค๋ฃน๋‹ˆ๋‹ค. ์—ฐ๊ตฌ์ง„์€ ๋ ˆ์ด์–ด ์˜คํ”„๋กœ๋”ฉ๊ณผ ์„ ์ œ์  ์ค‘๋‹จ(preemption)์ด LLM ์„ฑ๋Šฅ์— ๋ฏธ์น˜๋Š” ์˜ํ–ฅ์„ ๊ฒฝํ—˜์ ์œผ๋กœ ๋ถ„์„ํ•˜๊ณ , ๋ชจ๋ธ ์•„ํ‚คํ…์ฒ˜, ํฌ๊ธฐ, ํ•˜๋“œ์›จ์–ด ํ”Œ๋žซํผ์— ๋”ฐ๋ฅธ ๋น„์„ ํ˜•์  ์„ฑ๋Šฅ ์ €ํ•˜์™€ ์˜ค๋ฒ„ํ—ค๋“œ ๋ณ€ํ™”๋ฅผ ๊ทœ๋ช…ํ–ˆ์Šต๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ๋ถ„์„์„ ๋ฐ”ํƒ•์œผ๋กœ ๋ฏธ๋ž˜ LLM ์„œ๋น™ ์‹œ์Šคํ…œ์ด ๊ณ ๋ คํ•ด์•ผ ํ•  ํ•ต์‹ฌ ์š”์†Œ๋ฅผ ์ œ์‹œํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
์„œ๋กœ ๋‹ค๋ฅธ LLM ๋ชจ๋ธ๋“ค์ด GPU ๋ฉ”๋ชจ๋ฆฌ ์ œ์•ฝ์œผ๋กœ ์ธํ•œ ๋ ˆ์ด์–ด ์˜คํ”„๋กœ๋”ฉ ๋ฐ ์„ ์ œ์  ์ค‘๋‹จ์— ๋Œ€ํ•ด ๋ชจ๋ธ๋ณ„๋กœ ๋งค์šฐ ์ƒ์ดํ•˜๊ณ  ๋น„์„ ํ˜•์ ์ธ ์„ฑ๋Šฅ ๋ฏผ๊ฐ๋„๋ฅผ ๋ณด์ž…๋‹ˆ๋‹ค.
โ€ข
์„ ์ œ์  ์ค‘๋‹จ ์‹œ ๋ฐœ์ƒํ•˜๋Š” ์˜ค๋ฒ„ํ—ค๋“œ๋Š” ์ฃผ๋กœ ๋ชจ๋ธ ์ƒํƒœ ์žฌ๋กœ๋“œ์—์„œ ๊ธฐ์ธํ•˜๋ฉฐ, ํ‚ค-๊ฐ’ ์บ์‹œ ์ „์†ก๋ณด๋‹ค ๋” ํฐ ์˜ํ–ฅ์„ ๋ฏธ์น˜๊ณ  ๋ชจ๋ธ ๋ฐ ํ•˜๋“œ์›จ์–ด์— ๋”ฐ๋ผ ํฌ๊ฒŒ ๋‹ฌ๋ผ์ง‘๋‹ˆ๋‹ค.
โ€ข
์‹œํ€€์Šค ๊ธธ์ด์™€ ์ธํ„ฐ์ปค๋„ฅํŠธ ๋Œ€์—ญํญ์€ ๋ฐ์ดํ„ฐ ์ด๋™ ๋ฐ ์‹คํ–‰ ๋น„ํšจ์œจ์„ฑ์„ ์ฆํญ์‹œํ‚ค๋ฏ€๋กœ ์Šค์ผ€์ค„๋Ÿฌ ์„ค๊ณ„ ์‹œ ๋ฐ˜๋“œ์‹œ ๊ณ ๋ ค๋˜์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
๐Ÿ‘
Made with Slashpage