This page organizes papers related to artificial intelligence published around the world. This page is summarized using Google Gemini and is operated on a non-profit basis. The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.
This paper identifies temporal oscillation in the Diffusion Large-Scale Language Model (dLLM) and proposes two methods to address it. dLLM often generates text where intermediate predictions are more accurate than the final output. To address this issue, we leverage temporal consistency. First, we propose a Temporal Self-Consistency Voting technique that aggregates intermediate predictions without training and selects the most consistent output. Second, we propose a Temporal Consistency Reinforcement method that utilizes Temporal Semantic Entropy (TSE) to enhance the stability of the generation. The effectiveness of the proposed methodology is demonstrated on various benchmarks, demonstrating a 24.7% improvement over the existing dLLM on the Countdown dataset.
Takeaways, Limitations
•
Takeaways:
◦
Emphasizes the importance of temporal dynamics of dLLM, that is, changes over time, and suggests new ways to utilize them.
◦
Temporal Self-Consistency Voting without training method leads to improved performance in the testing phase.
◦
Achieve meaningful performance improvements across multiple datasets while increasing generation stability through Temporal Consistency Reinforcement.
•
Limitations:
◦
There may be insufficient description of the specific model structure or implementation method (based solely on the abstract).
◦
Additional comparative analysis with other existing dLLM models may be required.
◦
Further research is needed on the generalizability of new indicators such as TSE and their extensibility to other models.
