Sign In

On the Sample Complexity of Differentially Private Policy Optimization

์ž‘์„ฑ์ž
  • Haebom
์นดํ…Œ๊ณ ๋ฆฌ
Empty

์ €์ž

Yi He, Xingyu Zhou

๐Ÿ’ก ๊ฐœ์š”

๋ณธ ๋…ผ๋ฌธ์€ ๊ฐ•ํ™”ํ•™์Šต์˜ ํ•ต์‹ฌ ๊ธฐ๋ฒ•์ธ ์ •์ฑ… ์ตœ์ ํ™”(PO)์˜ ์ฐจ๋ถ„ ํ”„๋ผ์ด๋ฒ„์‹œ(DP) ์ƒ˜ํ”Œ ๋ณต์žก์„ฑ์„ ์ด๋ก ์ ์œผ๋กœ ํƒ๊ตฌํ•ฉ๋‹ˆ๋‹ค. ์—ฐ๊ตฌ์ง„์€ PO์— ์ ํ•ฉํ•œ DP ์ •์˜๋ฅผ ์ƒˆ๋กญ๊ฒŒ ์ œ์•ˆํ•˜๊ณ , ์ •์ฑ… ๊ฒฝ์‚ฌ(PG) ๋ฐ ์ž์—ฐ ์ •์ฑ… ๊ฒฝ์‚ฌ(NPG)์™€ ๊ฐ™์€ ๋‹ค์–‘ํ•œ PO ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ์ƒ˜ํ”Œ ๋ณต์žก์„ฑ์„ DP ์ œ์•ฝ ํ•˜์—์„œ ๋ถ„์„ํ•ฉ๋‹ˆ๋‹ค. ์ด๋ฅผ ํ†ตํ•ด ํ”„๋ผ์ด๋ฒ„์‹œ ๋น„์šฉ์ด ์ƒ˜ํ”Œ ๋ณต์žก์„ฑ์˜ ๋‚ฎ์€ ์ฐจํ•ญ์œผ๋กœ ๋‚˜ํƒ€๋‚  ์ˆ˜ ์žˆ์œผ๋ฉฐ, ํ”„๋ผ์ด๋ฒ„์‹œ๋ฅผ ๋ณดํ˜ธํ•˜๋Š” PO ์•Œ๊ณ ๋ฆฌ์ฆ˜ ์„ค๊ณ„์— ๋Œ€ํ•œ ์‹ค์งˆ์ ์ธ ํ†ต์ฐฐ๋ ฅ์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”‘ ์‹œ์‚ฌ์  ๋ฐ ํ•œ๊ณ„

โ€ข
์˜จ-ํด๋ฆฌ์‹œ ํ•™์Šต์˜ ํŠน์„ฑ๊ณผ ํ”„๋ผ์ด๋ฒ„์‹œ ๋‹จ์œ„ ์ •์˜์˜ ๋ณต์žก์„ฑ์„ ๊ณ ๋ คํ•œ ์ƒˆ๋กœ์šด ์ฐจ๋ถ„ ํ”„๋ผ์ด๋ฒ„์‹œ ์ •์˜๋ฅผ ์ œ์‹œํ•˜์—ฌ, ํ”„๋ผ์ด๋ฒ„์‹œ๋ฅผ ๋ณดํ˜ธํ•˜๋Š” ์ •์ฑ… ์ตœ์ ํ™” ์—ฐ๊ตฌ์˜ ๊ธฐ๋ฐ˜์„ ๋งˆ๋ จํ–ˆ์Šต๋‹ˆ๋‹ค.
โ€ข
๋‹ค์–‘ํ•œ ์ •์ฑ… ์ตœ์ ํ™” ์•Œ๊ณ ๋ฆฌ์ฆ˜์— ๋Œ€ํ•œ ํ†ต์ผ๋œ ๋ถ„์„ ํ”„๋ ˆ์ž„์›Œํฌ๋ฅผ ํ†ตํ•ด, ํ”„๋ผ์ด๋ฒ„์‹œ ๋น„์šฉ์ด ์ƒ˜ํ”Œ ๋ณต์žก์„ฑ์— ๋ฏธ์น˜๋Š” ์˜ํ–ฅ์„ ์ •๋Ÿ‰์ ์œผ๋กœ ์ดํ•ดํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
โ€ข
์ œ์‹œ๋œ ์ด๋ก ์  ๊ฒฐ๊ณผ๋Š” ์‹ค์ œ ํ”„๋ผ์ด๋ฒ„์‹œ ๋ณดํ˜ธ ๊ฐ•ํ™”ํ•™์Šต ์•Œ๊ณ ๋ฆฌ์ฆ˜ ์„ค๊ณ„ ์‹œ ์‹ค์งˆ์ ์ธ ๊ฐ€์ด๋“œ๋ผ์ธ์„ ์ œ๊ณตํ•˜๋ฉฐ, ํ”„๋ผ์ด๋ฒ„์‹œ์™€ ์„ฑ๋Šฅ ์‚ฌ์ด์˜ ๊ท ํ˜•์ ์„ ์ฐพ๋Š” ๋ฐ ๊ธฐ์—ฌํ•ฉ๋‹ˆ๋‹ค.
โ€ข
๋ณธ ์—ฐ๊ตฌ๋Š” ํŠน์ • DP ์ •์˜ ๋ฐ ์•Œ๊ณ ๋ฆฌ์ฆ˜์— ๊ตญํ•œ๋œ ๋ถ„์„์œผ๋กœ, ํ–ฅํ›„ ๋”์šฑ ์ผ๋ฐ˜์ ์ธ DP ์ •์˜์™€ ๋‹ค์–‘ํ•œ ๊ฐ•ํ™”ํ•™์Šต ์•Œ๊ณ ๋ฆฌ์ฆ˜์— ๋Œ€ํ•œ ํ™•์žฅ ์—ฐ๊ตฌ๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.
๐Ÿ‘
Made with Slashpage