Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

On the optimization dynamics of RLVR: Gradient gap and step size thresholds

Synthetic Series-Symbol Data Generation for Time Series Foundation Models

Learning Neural Exposure Fields for View Synthesis

Fewer Weights, More Problems: A Practical Attack on LLM Pruning

Can Lessons From Human Teams Be Applied to Multi-Agent Systems? The Role of Structure, Diversity, and Interaction Dynamics

Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation

GyroSwin: 5D Surrogates for Gyrokinetic Plasma Turbulence Simulations

Online Rubrics Elicitation from Pairwise Comparisons

Scalable multilingual PII annotation for responsible AI in LLMs

Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods

Reinforce-Ada: An Adaptive Sampling Framework for Reinforce-Style LLM Training

Revealing Interconnections between Diseases: from Statistical Methods to Large Language Models

P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs

MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering

Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention

Spatiotemporal Forecasting as Planning: A Model-Based Reinforcement Learning Approach with Generative World Models

6G-Enabled Digital Twin Framework for Real-Time Cyber-Physical Systems: An Experimental Validation with Industrial Bearing Fault Detection

InstructPLM-mu: 1-Hour Fine-Tuning of ESM2 Beats ESM3 in Protein Mutation Predictions

Prompt-Aware Scheduling for Low-Latency LLM Serving

Learning Robust Diffusion Models from Imprecise Supervision

CLARITY: Clinical Assistant for Routing, Inference, and Triage

Nav-EE: Navigation-Guided Early Exiting for Efficient Vision-Language Models in Autonomous Driving

Machine Learning for Detection and Analysis of Novel LLM Jailbreaks

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

TimeScope: Towards Task-Oriented Temporal Grounding In Long Videos

Editable Noise Map Inversion: Encoding Target-image into Noise For High-Fidelity Image Manipulation

ClustRecNet: A Novel End-to-End Deep Learning Framework for Clustering Algorithm Recommendation

Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithm

InfiR2: A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models

Benchmarking and Mitigate Sycophancy in Medical Vision-Language Models

CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics

A Mega-Study of Digital Twins Reveals Strengths, Weaknesses and Opportunities for Further Improvement

Robust LLM Training Infrastructure at ByteDance

RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation

MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment

Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal

Individual utilities of life satisfaction reveal inequality aversion unrelated to political alignment

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens

From Federated Learning to X-Learning: Breaking the Barriers of Decentrality Through Random Walks

Latent Variable Modeling in Multi-Agent Reinforcement Learning via Expectation-Maximization for UAV-Based Wildlife Protection

Quantifying Label-Induced Bias in Large Language Model Self- and Cross-Evaluations

Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices

On Task Vectors and Gradients

On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting

LATTE: Learning Aligned Transactions and Textual Embeddings for Bank Clients

ACD-CLIP: Decoupling Representation and Dynamic Fusion for Zero-Shot Anomaly Detection

Neural Beam Field for Spatial Beam RSRP Prediction

AMFT: Aligning LLM Reasoners by Meta-Learning the Optimal Imitation-Exploration Balance

Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling

VAGPO: Vision-augmented Asymmetric Group Preference Optimization for Graph Routing Problems

TriP-LLM: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detection

Enabling Self-Improving Agents to Learn at Test Time With Human-In-The-Loop Guidance

Preprint: Poster: Did I Just Browse A Website Written by LLMs?

Site-Level Fine-Tuning with Progressive Layer Freezing: Towards Robust Prediction of Bronchopulmonary Dysplasia from Day-1 Chest Radiographs in Extremely Preterm Infants

AirScape: An Aerial Generative World Model with Motion Controllability

Tuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training

EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework

Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments with a Hierarchical Spatial-Cognition Long-Short Memory System

Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspective

Interpretable and Granular Video-Based Quantification of Motor Characteristics from the Finger Tapping Test in Parkinson Disease

Bures-Wasserstein Flow Matching for Graph Generation

Symmetry in Neural Network Parameter Spaces

CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models

Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs

AD-EE: Early Exiting for Fast and Reliable Vision-Language Models in Autonomous Driving

Robustness in Both Domains: CLIP Needs a Robust Text Encoder

Elicit and Enhance: Advancing Multimodal Reasoning in Medical Scenarios

Any-to-Bokeh: Arbitrary-Subject Video Refocusing with Video Diffusion Model

An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks

FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information

Beyond Demonstrations: Dynamic Vector Construction from Latent Representations

DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaboration

Sequential Monte Carlo for Policy Optimization in Continuous POMDPs

Game of Trust: How Trustworthy Does Your Blockchain Think You Are?

Collaborative Unlabeled Data Optimization

Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Features

System Prompt Optimization with Meta-Learning

Cognitio Emergens: Agency, Dimensions, and Dynamics in Human-AI Knowledge Co-Creation

Multimodal Language Models See Better When They Look Shallower

Exploring human-SAV interaction using LLMs: The impact of psychological factors on user experience

On Developers' Self-Declaration of AI-Generated Code: An Analysis of Practices

Diffusion Generative Recommendation with Continuous Tokens

TARO: Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning for Synchronized Video-to-Audio Synthesis

DeepOHeat-v1: Efficient Operator Learning for Fast and Trustworthy Thermal Simulation and Optimization in 3D-IC Design

Brain2Text Decoding Model Reveals the Neural Mechanisms of Visual Semantic Processing

Issue Localization via LLM-Driven Iterative Code Graph Searching

CCDP: Composition of Conditional Diffusion Policies with Guided Sampling

Measuring directional bias amplification in image captions using predictability

Contrastive Learning Augmented Social Recommendations

WyckoffDiff -- A Generative Diffusion Model for Crystal Symmetry

Solving Linear-Gaussian Bayesian Inverse Problems with Decoupled Diffusion Sequential Monte Carlo

RadVLM: A Multitask Conversational Vision-Language Model for Radiology

IG-MCTS: Human-in-the-Loop Cooperative Navigation under Incomplete Information

OrcaLoca: An LLM Agent Framework for Software Issue Localization

Enabling Population-Level Parallelism in Tree-Based Genetic Programming for GPU Acceleration

AD-LLM: Benchmarking Large Language Models for Anomaly Detection

Preference Discerning with LLM-Enhanced Generative Retrieval

SwarmGPT: Combining Large Language Models with Safe Motion Planning for Drone Swarm Choreography

Generate Any Scene: Scene Graph Driven Data Synthesis for Visual Generation Training

Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence

Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices

Created by

Haebom

저자

Fahao Chen, Jie Wan, Peng Li, Zhou Su, Dongxiao Yu

개요

FLUX는 제한된 컴퓨팅 리소스를 가진 참여자들(예: 소비자급 GPU) 간의 MoE 기반 LLM의 연합 미세 조정을 가능하게 하도록 설계된 시스템입니다. 시간-정확도를 최소화하는 것을 목표로 하며, 세 가지 주요 혁신을 도입했습니다. (1) 최소한의 오버헤드로 전문가 활성화를 추정하는 양자화 기반 로컬 프로파일링, (2) 정확도를 유지하면서 리소스 소비를 줄이는 적응형 레이어별 전문가 병합, (3) 튜닝 및 비 튜닝 전문가 간의 균형을 맞추기 위한 탐색-활용 전략을 사용하는 동적 전문가 역할 할당입니다.

시사점, 한계점

•

시사점:

◦

FLUX는 MoE 기반 LLM의 연합 미세 조정을 위한 효과적인 시스템을 제공하여, 제한된 리소스를 가진 환경에서도 LLM 학습을 가능하게 합니다.

◦

양자화, 적응형 전문가 병합, 동적 전문가 역할 할당 등의 혁신적인 기법을 통해 시간-정확도를 크게 향상시켰습니다 (최대 4.75배 속도 향상).

◦

LLaMA-MoE 및 DeepSeek-MoE 모델과 다양한 벤치마크 데이터셋을 사용한 광범위한 실험을 통해 기존 방법보다 우수함을 입증했습니다.

•

한계점:

◦

논문에서 구체적인 시스템 가정이나 제약 조건에 대한 자세한 내용은 제시되지 않았습니다.

◦

모델 크기, 데이터셋 특성, 참여자 리소스의 다양성 등 다른 요인들에 대한 FLUX의 성능 변화에 대한 분석이 부족합니다.

◦

구체적인 구현 세부 사항이나, 다른 종류의 MoE 모델에 대한 일반화 가능성에 대한 추가 연구가 필요합니다.

Made with Slashpage