Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models

ManiAgent: An Agentic Framework for General Robotic Manipulation

AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Model

ParsVoice: A Large-Scale Multi-Speaker Persian Speech Corpus for Text-to-Speech Synthesis

Optimally Deep Networks -- Adapting Model Depth to Datasets for Superior Efficiency

BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions

AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generation

Towards Safe Maneuvering of Double-Ackermann-Steering Robots with a Soft Actor-Critic Framework

The Algorithmic Regulator

HccePose(BF): Predicting Front & Back Surfaces to Construct Ultra-Dense 2D-3D Correspondences for Pose Estimation

ICL-Router: In-Context Learned Model Representations for LLM Routing

CrisiText: A dataset of warning messages for LLM training in emergency communication

GTCN-G: A Residual Graph-Temporal Fusion Network for Imbalanced Intrusion Detection (Preprint)

Scalable Policy-Based RL Algorithms for POMDPs

OptiFLIDS: Optimized Federated Learning for Energy-Efficient Intrusion Detection in IoT

Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain

AgentBuilder: Exploring Scaffolds for Prototyping User Experiences of Interface Agents

Neon: Negative Extrapolation From Self-Training Improves Image Generation

Triplet-Structured Knowledge Integration for Multi-Turn Medical Reasoning

General Exploratory Bonus for Optimistic Exploration in RLHF

PolySim: Bridging the Sim-to-Real Gap for Humanoid Control via Multi-Simulator Dynamics Randomization

Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoning

Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs

Understanding Language Prior to LVLMs by Contrasting Chain-of-Embedding

Large language models management of medications: three performance analyzes

Responsible AI Technical Report

A Longitudinal Randomized Control Study of Companion Chatbot Use: Anthropomorphism and Its Mediating Role on Social Impacts

SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer

TISDiSS: A Training-Time and Inference-Time Scalable Framework for Discriminative Source Separation

Efficient and Versatile Model for Multilingual Information Retrieval of Islamic Text: Development and Deployment in Real-World Scenarios

StegOT: Trade-offs in Steganography via Optimal Transport

General Demographic Foundation Models for Enhancing Predictive Performance Across Diseases and Populations

Attention as an Adaptive Filter

Diffusion Language Models Know the Answer Before Decoding

NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows

Hard Examples Are All You Need: Maximizing GRPO Post-Training Under Annotation Budgets

AdaptJobRec: Enhancing Conversational Career Recommendation through an LLM-Powered Agentic System

An Introduction to Sliced Optimal Transport

EMSEdit: Efficient Multi-Step Meta-Learning-based Model Editing

DMSC: Dynamic Multi-Scale Coordination Framework for Time Series Forecasting

MmWave Radar-Based Non-Line-of-Sight Pedestrian Localization at T-Junctions Utilizing Road Layout Extraction via Camera

Capturing More: Learning Multi-Domain Representations for Robust Online Handwriting Verification

A Cooperative Approach for Knowledge-based Business Process Design in a Public Authority

Finding Dori: Memorization in Text-to-Image Diffusion Models Is Not Local

Feature Distillation is the Better Choice for Model-Heterogeneous Federated Learning

Knowledge Fusion via Bidirectional Information Aggregation

LearnLens: LLM-Enabled Personalized, Curriculum-Grounded Feedback with Educators in the Loop

Dual Perspectives on Non-Contrastive Self-Supervised Learning

SAFER: Probing Safety in Reward Models with Sparse Autoencoder

Inverse Design in Nanophotonics via Representation Learning

SPADE: Spatial Transcriptomics and Pathology Alignment Using a Mixture of Data Experts for an Expressive Latent Space

VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents

Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series

BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models

Uncertainty Estimation on Graphs with Structure Informed Stochastic Partial Differential Equations

EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning

Can LLMs Reason Structurally? An Evaluation via the Lens of Data Structures

Finite Sample Analysis of Linear Temporal Difference Learning with Arbitrary Features

Leveraging Importance Sampling to Detach Alignment Modules from Large Language Models

Protein Design with Dynamic Protein Vocabulary

Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator

Steering Large Language Models for Machine Translation Personalization

AsynFusion: Towards Asynchronous Latent Consistency Models for Decoupled Whole-Body Audio-Driven Avatars

Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learning

Fixed Point Explainability

Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking

MobileCity: An Efficient Framework for Large-Scale Urban Behavior Simulation

A Customized SAT-based Solver for Graph Coloring

Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances

FOCUS on Contamination: A Geospatial Deep Learning Framework with a Noise-Aware Loss for Surface Water PFAS Prediction

ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization

Query Brand Entity Linking in E-Commerce Search

AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvement

GraphRAG under Fire

Polynomial-Time Algorithms for Fair Orientations of Chores

CiteBART: Learning to Generate Citations for Local Citation Recommendations

Reframing Image Difference Captioning with BLIP2IDC and Synthetic Augmentation

Multi-View Majority Vote Learning Algorithms: Direct Minimization of PAC-Bayesian Bounds

COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences

Generative AI for Requirements Engineering: A Systematic Literature Review

Assessing Latency in ASR Systems: A Methodological Perspective for Real-Time Use

ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training

Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions

Offline Fictitious Self-Play for Competitive Games

When “Competency” in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers

Can ChatGPT support software verification?

Optimized Layerwise Approximation for Efficient Private Inference on Fully Homomorphic Encryption

DRIFT: Decompose, Retrieve, Illustrate, then Formalize Theorems

Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoning

Concise Reasoning in the Lens of Lagrangian Optimization

Humanoid Artificial Consciousness Designed with Large Language Model Based on Psychoanalysis and Personality Theory

TripScore: Benchmarking and rewarding real-world travel planning with fine-grained evaluation

Agent Learning via Early Experience

L2M-AID: Autonomous Cyber-Physical Defense by Fusing Semantic Reasoning of Large Language Models with Multi-Agent Reinforcement Learning (Preprint)

Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment

Large Language Models in Operations Research: Methods, Applications, and Challenges

MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE

Strategic Tradeoffs Between Humans and AI in Multi-Agent Bargaining

MapAgent: A Hierarchical Agent for Geospatial Reasoning with Dynamic Map Tool Integration

Dual Perspectives on Non-Contrastive Self-Supervised Learning

Created by

Haebom

Author

Jean Ponce (ENS-PSL, NYU), Basile Terver (FAIR, WILLOW), Martial Hebert (CMU), Michael Arbel (Thoth)

Outline

This paper analyzes the "stop gradient" and "exponential moving average" iterative procedures used to prevent representation collapse in non-contrast approaches to self-supervised learning from an optimization and dynamical systems perspective. We show that these procedures prevent representation collapse even without optimizing the original objective function or another smooth function. Using a dynamical systems perspective in a linear context, we prove that minimizing the original objective function without "stop gradient" or "exponential moving average" always leads to representation collapse. Furthermore, we explicitly characterize the equilibrium points of the dynamical systems associated with these two procedures in the linear setting and show that they are generally asymptotically stable.

Takeaways, Limitations

•

We theoretically show that "stop gradient" and "exponential moving average" do not actually optimize the original objective function, but are effective in preventing representation collapse.

•

We provide a rigorous mathematical proof of the effects of "stop gradient" and "exponential moving average" in a linear environment.

•

Understand the stability of a dynamic system by analyzing its equilibrium points in relation to "stop gradient" and "exponential moving average."

•

This study focuses on theoretical analysis, and further research is needed on generalization to other types of self-supervised learning techniques or nonlinear environments.

•

The theoretical results are verified through experiments using real and synthetic data.

View PDF

Made with Slashpage