This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
This paper highlights the importance of well-designed prompts for improving the inference capability of large-scale language models (LLMs) and aligning outputs with task requirements across diverse domains. Existing prompt optimization methods rely heavily on external references, such as correct answers or human intervention, limiting their applicability to real-world scenarios. To address this, this paper proposes Self-Supervised Prompt Optimization (SPO), a cost-effective prompt optimization framework that does not require external references. SPO derives evaluation and optimization signals from LLM output comparisons, selects superior prompts through pairwise output comparisons using an LLM evaluator, and aligns outputs with task requirements using an LLM optimizer. Experimental results demonstrate that SPO achieves comparable or superior performance compared to existing methods while significantly reducing costs (1.1% to 5.6%) and the number of samples (3).
Takeaways, Limitations
•
Takeaways:
◦
A new method (SPO) is presented to efficiently optimize prompts without external references.
◦
Achieve superior performance with significantly lower cost and sample count than existing methods.
◦
Applicability to various tasks (closed and open).
◦
Leverage LLM's own capabilities to automate prompt optimization.
•
Limitations:
◦
May depend on the performance of the LLM evaluator and optimizer.
◦
It may be a method optimized for a specific LLM.
◦
Generalization performance verification is needed for various domains and task types.
◦
Errors by LLM evaluators and optimizers can affect the final results.
