Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

Permissioned LLMs: Enforcing Access Control in Large Language Models

Created by
  • Haebom

Author

Bargav Jayaraman, Virendra J. Marathe, Hamid Mozaffari, William F. Shen, Krishnaram Kenthapadi

Outline

This paper proposes "Permissioned LLMs (PermLLM)," a new LLM class that enforces organizational data access control structures in query responses to address the challenges that arise when large-scale language models (LLMs) trained on isolated and isolated organizational data in enterprise environments serve users with diverse access privileges. We present abstractions to demonstrate the correct enforcement of access control in PermLLM, relevant response concepts, and a new metric, access advantage, to evaluate the effectiveness of the PermLLM mechanism. Furthermore, we introduce three new PermLLM mechanisms based on parameter efficient fine-tuning and present two implementations of access advantage: the Domain Distinguishability Index (DDI) based on membership inference attacks and the Utility Gap Index (UGI) based on LLM utility assessment. We extensively experiment with the effectiveness of the PermLLM mechanism and evaluate the effectiveness of the DDI and UGI metrics using five publicly available datasets: GPQA, RCV1, SimpleQA, WMDP, and PubMedQA.

Takeaways, Limitations

Takeaways:
A novel approach to addressing data access control challenges in LLMs in corporate environments.
Development of formal abstractions, relevant response concepts, and access benefit metrics to ensure and evaluate the correct operation of PermLLM.
Proposing three PermLLM mechanisms based on parameter efficient fine-tuning.
Development of DDI and UGI metrics to measure access benefits.
The effectiveness of the PermLLM mechanism was verified through extensive experiments using various datasets.
Limitations:
Further research is needed on the practical implementation and deployment of the proposed PermLLM mechanism.
Further evaluation is needed to determine how generalizable the DDI and UGI metrics are to different data types and LLM models.
Further research is needed on how to balance the performance of the PermLLM mechanism with access control.
Lack of information on testing and application cases in real business environments.
👍