Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?

Created by
  • Haebom

Author

Hua Shen, Nicholas Clark, Tanushree Mitra

Outline

This paper addresses the limitations of existing research in assessing the value alignment of large-scale language models (LLMs) and proposes ValueActionLens, a novel evaluation framework that considers the "value-action gap." Leveraging a dataset of 14,800 value-based actions across 12 cultures and 11 social topics, ValueActionLens assesses the alignment between LLMs' stated values and value-based actions using three metrics. Experimental results demonstrate that the alignment between LLMs' stated values and actions is suboptimal and varies significantly across contexts and models. Furthermore, we identify potential harms caused by value-action gaps and demonstrate the effectiveness of using inferential explanations to predict such gaps. In conclusion, we highlight the dangers of relying solely on stated values to predict LLM behavior and emphasize the importance of context-sensitive assessment of LLM values and value-action gaps.

Takeaways, Limitations

•
Takeaways:
◦
This demonstrates the limitations of simply considering stated values in evaluating the value alignment of LLMs.
◦
Introducing ValueActionLens, a new evaluation framework that takes into account the value-action gap phenomenon.
◦
To identify the potential harm caused by the value-action gap in LLM and to seek ways to mitigate it.
◦
Suggesting the possibility of improving the performance of predicting value-behavior gaps by utilizing inferential explanations.
◦
Emphasize the importance of context-aware LLM valuation.
•
Limitations:
◦
Further research is needed to explore the generalizability of the ValueActionLens framework.
◦
Further experiments are needed across different LLM models and in a wider range of situations.
◦
Further research is needed to determine the exact causes of the value-behavior gap and to find solutions.
◦
The dataset needs to be reviewed for cultural bias and the objectivity of social topic selection.
👍
Made with Slashpage