Daily Arxiv

世界中で発行される人工知能関連の論文をまとめるページです。
このページはGoogle Geminiを活用して要約し、非営利で運営しています。
論文の著作権は著者および関連機関にあり、共有する際は出典を明記してください。

Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models

ManiAgent: An Agentic Framework for General Robotic Manipulation

AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Model

ParsVoice: A Large-Scale Multi-Speaker Persian Speech Corpus for Text-to-Speech Synthesis

Optimally Deep Networks - Adapting Model Depth to Datasets for Superior Efficiency

BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions

AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generation

Towards Safe Maneuvering of Double-Ackermann-Steering Robots with a Soft Actor-Critic Framework

The Algorithmic Regulator

HccePose(BF): Predicting Front & Back Surfaces to Construct Ultra-Dense 2D-3D Correspondences for Pose Estimation

ICL-Router: In-Context Learned Model Representations for LLM Routing

CrisiText: A dataset of warning messages for LLM training in emergency communication

GTCN-G: A Residual Graph-Temporal Fusion Network for Imbalanced Intrusion Detection (Preprint)

Scalable Policy-Based RL Algorithms for POMDPs

OptiFLIDS: Optimized Federated Learning for Energy-Efficient Intrusion Detection in IoT

Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain

AgentBuilder: Exploring Scaffolds for Prototyping User Experiences of Interface Agents

Neon: Negative Extrapolation From Self-Training Improves Image Generation

Triplet-Structured Knowledge Integration for Multi-Turn Medical Reasoning

General Exploratory Bonus for Optimistic Exploration in RLHF

PolySim: Bridging the Sim-to-Real Gap for Humanoid Control via Multi-Simulator Dynamics Randomization

Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoning

Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs

Understanding Language Prior of LVLMs by Contrasting Chain-of-Embedding

Large language models management of medications: three performance analyses

Responsible AI Technical Report

A Longitudinal Randomized Control Study of Companion Chatbot Use: Anthropomorphism and Its Mediating Role on Social Impacts

SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer

TISDiSS: A Training-Time and Inference-Time Scalable Framework for Discriminative Source Separation

Efficient and Versatile Model for Multilingual Information Retrieval of Islamic Text: Development and Deployment in Real-World Scenarios

StegOT: Trade-offs in Steganography via Optimal Transport

General Demographic Foundation Models for Enhancing Predictive Performance Across Diseases and Populations

Attention as an Adaptive Filter

Diffusion Language Models Know the Answer Before Decoding

NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows

Hard Examples Are All You Need: Maximizing GRPO Post-Training Under Annotation Budgets

AdaptJobRec: Enhancing Conversational Career Recommendation through an LLM-Powered Agentic System

An Introduction to Sliced Optimal Transport

EMSEdit: Efficient Multi-Step Meta-Learning-based Model Editing

DMSC: Dynamic Multi-Scale Coordination Framework for Time Series Forecasting

MmWave Radar-Based Non-Line-of-Sight Pedestrian Localization at T-Junctions Utilizing Road Layout Extraction via Camera

Capturing More: Learning Multi-Domain Representations for Robust Online Handwriting Verification

A Cooperative Approach for Knowledge-based Business Process Design in a Public Authority

Finding Dori: Memorization in Text-to-Image Diffusion Models Is Not Local

Feature Distillation is the Better Choice for Model-Heterogeneous Federated Learning

Knowledge Fusion via Bidirectional Information Aggregation

LearnLens: LLM-Enabled Personalised, Curriculum-Grounded Feedback with Educators in the Loop

Dual Perspectives on Non-Contrastive Self-Supervised Learning

SAFER: Probing Safety in Reward Models with Sparse Autoencoder

Inverse Design in Nanophotonics via Representation Learning

SPADE: Spatial Transcriptomics and Pathology Alignment Using a Mixture of Data Experts for an Expressive Latent Space

VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents

Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series

BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models

Uncertainty Estimation on Graphs with Structure Informed Stochastic Partial Differential Equations

EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning

Can LLMs Reason Structurally? An Evaluation via the Lens of Data Structures

Finite Sample Analysis of Linear Temporal Difference Learning with Arbitrary Features

Leveraging Importance Sampling to Detach Alignment Modules from Large Language Models

Protein Design with Dynamic Protein Vocabulary

Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator

Steering Large Language Models for Machine Translation Personalization

AsynFusion: Towards Asynchronous Latent Consistency Models for Decoupled Whole-Body Audio-Driven Avatars

Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learning

Fixed Point Explainability

Time Travel is Cheating: Going Live with DeepFund for Real-Time Fund Investment Benchmarking

MobileCity: An Efficient Framework for Large-Scale Urban Behavior Simulation

A Customized SAT-based Solver for Graph Coloring

Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances

FOCUS on Contamination: A Geospatial Deep Learning Framework with a Noise-Aware Loss for Surface Water PFAS Prediction

ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization

Query Brand Entity Linking in E-Commerce Search

AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvement

GraphRAG under Fire

Polynomial-Time Algorithms for Fair Orientations of Chores

CiteBART: Learning to Generate Citations for Local Citation Recommendation

Reframing Image Difference Captioning with BLIP2IDC and Synthetic Augmentation

Multi-View Majority Vote Learning Algorithms: Direct Minimization of PAC-Bayesian Bounds

COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences

Generative AI for Requirements Engineering: A Systematic Literature Review

Assessing Latency in ASR Systems: A Methodological Perspective for Real-Time Use

ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training

Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions

Offline Fictitious Self-Play for Competitive Games

When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers

Can ChatGPT support software verification?

Optimized Layerwise Approximation for Efficient Private Inference on Fully Homomorphic Encryption

DRIFT: Decompose, Retrieve, Illustrate, then Formalize Theorems

Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoning

Concise Reasoning in the Lens of Lagrangian Optimization

Humanoid Artificial Consciousness Designed with Large Language Model Based on Psychoanalysis and Personality Theory

TripScore: Benchmarking and rewarding real-world travel planning with fine-grained evaluation

Agent Learning via Early Experience

L2M-AID: Autonomous Cyber-Physical Defense by Fusing Semantic Reasoning of Large Language Models with Multi-Agent Reinforcement Learning (Preprint)

Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment

Similarity Field Theory: A Mathematical Framework for Intelligence

Large Language Models in Operations Research: Methods, Applications, and Challenges

MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE

Strategic Tradeoffs Between Humans and AI in Multi-Agent Bargaining

MapAgent: A Hierarchical Agent for Geospatial Reasoning with Dynamic Map Tool Integration

MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE

Created by

Haebom

作者

Soheil Zibakhsh, Mohammad Samragh, Kumari Nishu, Lauren Hannah, Arnav Kundu, Minsik Cho

概要

大規模言語モデル（LLM）の生成品質は、推論時間シーケンスレベルのスケーリング方法（例えば、チェーンオブザフト）を使用して改善されることがよくあります。本稿では、トークンレベルで予測品質を向上させる補完的なフレームワークであるハイパーパラレルスケーリングを紹介します。ハイパーパラレルスケーリングは、モデル内の単一トークンに対する複数の出力提案を計算して集計します。この研究では、これをRoster of Experts（RoE）というMixture-of-Experts（MoE）モデルに実装しました。 RoEは、単一のMoEを動的MoEアンサンブルに変換する学習を必要としない推論アルゴリズムです。 RoEは、エキスパートルーティングメカニズムに制御された確率的要素を注入し、各トークンに対してさまざまなエキスパートをサンプリングし、より正確な最終予測のためにその出力を集約することができます。計算コストを克服するために、効率的な展開戦略とコンピュータとメモリのオーバーヘッドを最小限に抑える特別なKVキャッシュメカニズムを導入しました。たとえば、RoE を使用すると、7B MoE モデルが推論時に 30% 少ない計算量で 10.5B MoE モデルのパフォーマンスに匹敵します。これらの利点は、モデルパラメータを微調整することなく得ることができます。

Takeaways、Limitations

•

Takeaways:

◦

推論時にトークンレベルで予測品質を向上させる新しいフレームワークであるハイパーパラレルスケーリングの提示

◦

Mixture-of-Experts (MoE)モデルのためのトレーニングフリーの推論アルゴリズムであるRoster of Experts (RoE)の開発

◦

効率的な配置戦略とKVキャッシングメカニズムによる計算とメモリオーバーヘッドの削減

◦

モデルの微調整なしに、より小さなモデルでより大きなモデルのパフォーマンスを達成可能

•

Limitations:

◦

具体的な実験結果や性能比較の詳細は論文には記載されていない

◦

RoEの一般的なモデルの適用性については言及しない

◦

他のスケーリング技術との互換性と相乗効果の議論は存在しない

Made with Slashpage