Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

On the optimization dynamics of RLVR: Gradient gap and step size thresholds

Synthetic Series-Symbol Data Generation for Time Series Foundation Models

Learning Neural Exposure Fields for View Synthesis

Fewer Weights, More Problems: A Practical Attack on LLM Pruning

Can Lessons From Human Teams Be Applied to Multi-Agent Systems? The Role of Structure, Diversity, and Interaction Dynamics

Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation

GyroSwin: 5D Surrogates for Gyrokinetic Plasma Turbulence Simulations

Online Rubrics Elicitation from Pairwise Comparisons

Scalable multilingual PII annotation for responsible AI in LLMs

Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods

Reinforce-Ada: An Adaptive Sampling Framework for Reinforce-Style LLM Training

Revealing Interconnections between Diseases: from Statistical Methods to Large Language Models

P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs

MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering

Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention

Spatiotemporal Forecasting as Planning: A Model-Based Reinforcement Learning Approach with Generative World Models

6G-Enabled Digital Twin Framework for Real-Time Cyber-Physical Systems: An Experimental Validation with Industrial Bearing Fault Detection

InstructPLM-mu: 1-Hour Fine-Tuning of ESM2 Beats ESM3 in Protein Mutation Predictions

Prompt-Aware Scheduling for Low-Latency LLM Serving

Learning Robust Diffusion Models from Imprecise Supervision

CLARITY: Clinical Assistant for Routing, Inference, and Triage

Nav-EE: Navigation-Guided Early Exiting for Efficient Vision-Language Models in Autonomous Driving

Machine Learning for Detection and Analysis of Novel LLM Jailbreaks

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

TimeScope: Towards Task-Oriented Temporal Grounding In Long Videos

Editable Noise Map Inversion: Encoding Target-image into Noise For High-Fidelity Image Manipulation

ClustRecNet: A Novel End-to-End Deep Learning Framework for Clustering Algorithm Recommendation

Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithm

InfiR2: A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models

Benchmarking and Mitigate Sycophancy in Medical Vision-Language Models

CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics

A Mega-Study of Digital Twins Reveals Strengths, Weaknesses and Opportunities for Further Improvement

Robust LLM Training Infrastructure at ByteDance

RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation

MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment

Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal

Individual utilities of life satisfaction reveal inequality aversion unrelated to political alignment

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens

From Federated Learning to X-Learning: Breaking the Barriers of Decentrality Through Random Walks

Latent Variable Modeling in Multi-Agent Reinforcement Learning via Expectation-Maximization for UAV-Based Wildlife Protection

Quantifying Label-Induced Bias in Large Language Model Self- and Cross-Evaluations

Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices

On Task Vectors and Gradients

On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting

LATTE: Learning Aligned Transactions and Textual Embeddings for Bank Clients

ACD-CLIP: Decoupling Representation and Dynamic Fusion for Zero-Shot Anomaly Detection

Neural Beam Field for Spatial Beam RSRP Prediction

AMFT: Aligning LLM Reasoners by Meta-Learning the Optimal Imitation-Exploration Balance

Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling

VAGPO: Vision-augmented Asymmetric Group Preference Optimization for Graph Routing Problems

TriP-LLM: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detection

Enabling Self-Improving Agents to Learn at Test Time With Human-In-The-Loop Guidance

Preprint: Poster: Did I Just Browse A Website Written by LLMs?

Site-Level Fine-Tuning with Progressive Layer Freezing: Towards Robust Prediction of Bronchopulmonary Dysplasia from Day-1 Chest Radiographs in Extremely Preterm Infants

AirScape: An Aerial Generative World Model with Motion Controllability

Tuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training

EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework

Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments with a Hierarchical Spatial-Cognition Long-Short Memory System

Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspective

Interpretable and Granular Video-Based Quantification of Motor Characteristics from the Finger Tapping Test in Parkinson Disease

Bures-Wasserstein Flow Matching for Graph Generation

Symmetry in Neural Network Parameter Spaces

CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models

Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs

AD-EE: Early Exiting for Fast and Reliable Vision-Language Models in Autonomous Driving

Robustness in Both Domains: CLIP Needs a Robust Text Encoder

Elicit and Enhance: Advancing Multimodal Reasoning in Medical Scenarios

Any-to-Bokeh: Arbitrary-Subject Video Refocusing with Video Diffusion Model

An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks

FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information

Beyond Demonstrations: Dynamic Vector Construction from Latent Representations

DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaboration

Sequential Monte Carlo for Policy Optimization in Continuous POMDPs

Game of Trust: How Trustworthy Does Your Blockchain Think You Are?

Collaborative Unlabeled Data Optimization

Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Features

System Prompt Optimization with Meta-Learning

Cognitio Emergens: Agency, Dimensions, and Dynamics in Human-AI Knowledge Co-Creation

Multimodal Language Models See Better When They Look Shallower

Exploring human-SAV interaction using LLMs: The impact of psychological factors on user experience

On Developers' Self-Declaration of AI-Generated Code: An Analysis of Practices

Diffusion Generative Recommendation with Continuous Tokens

TARO: Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning for Synchronized Video-to-Audio Synthesis

DeepOHeat-v1: Efficient Operator Learning for Fast and Trustworthy Thermal Simulation and Optimization in 3D-IC Design

Brain2Text Decoding Model Reveals the Neural Mechanisms of Visual Semantic Processing

Issue Localization via LLM-Driven Iterative Code Graph Searching

CCDP: Composition of Conditional Diffusion Policies with Guided Sampling

Measuring directional bias amplification in image captions using predictability

Contrastive Learning Augmented Social Recommendations

WyckoffDiff -- A Generative Diffusion Model for Crystal Symmetry

Solving Linear-Gaussian Bayesian Inverse Problems with Decoupled Diffusion Sequential Monte Carlo

RadVLM: A Multitask Conversational Vision-Language Model for Radiology

IG-MCTS: Human-in-the-Loop Cooperative Navigation under Incomplete Information

OrcaLoca: An LLM Agent Framework for Software Issue Localization

Enabling Population-Level Parallelism in Tree-Based Genetic Programming for GPU Acceleration

AD-LLM: Benchmarking Large Language Models for Anomaly Detection

Preference Discerning with LLM-Enhanced Generative Retrieval

SwarmGPT: Combining Large Language Models with Safe Motion Planning for Drone Swarm Choreography

Generate Any Scene: Scene Graph Driven Data Synthesis for Visual Generation Training

Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence

Robust LLM Training Infrastructure at ByteDance

Created by

Haebom

저자

Borui Wan, Gaohong Liu, Zuquan Song, Jun Wang, Yun Zhang, Guangming Sheng, Shuguang Wang, Houmin Wei, Chenyuan Wang, Weiqiang Lou, Xi Yang, Mofan Zhang, Kaihua Jiang, Cheng Ren, Xiaoyun Zhi, Menghan Yu, Zhe Nan, Zhuolin Zheng, Baoquan Zhong, Qinlong Wang, Huan Yu, Jinxin Chi, Wang Zhang, Yuhan Li, Zixian Du, Sida Zhao, Yongqiang Zhang, Jingzhe Tang, Zherui Liu, Chuan Wu, Yanghua Peng, Haibin Lin, Wencong Xiao, Xin Liu, Liang Xiang

개요

대규모 언어 모델(LLM)의 훈련 규모가 증가함에 따라 훈련 안정성에 대한 문제가 발생하며, 이는 훈련 중단, 효율적인 오류 진단, 효과적인 오류 허용을 필요로 한다. 본 논문은 LLM의 안정적인 훈련을 위한 대규모 GPU 인프라 관리 시스템인 ByteRobust를 제시한다. ByteRobust는 LLM 훈련 과정의 특징을 활용하여 오류 감지 및 복구를 우선시하며, 데이터 기반 접근 방식을 통해 높은 수준의 오류 허용, 신속한 오류 구분 및 위치 파악을 가능하게 한다. 20만 개 이상의 GPU가 탑재된 프로덕션 GPU 플랫폼에 배포되어 9,600개의 GPU에서 3개월간의 훈련 작업에서 97%의 ETTR을 달성했다.

시사점, 한계점

•

시사점:

◦

LLM 훈련의 안정성을 위한 효과적인 GPU 인프라 관리 시스템 제시

◦

오류 감지 및 복구를 최우선으로 하는 접근 방식

◦

대규모 GPU 환경에서 높은 수준의 오류 허용 및 효율적인 훈련 가능성 입증

◦

실제 프로덕션 환경에서의 성공적인 배포 및 높은 ETTR 달성

•

한계점:

◦

논문에서 구체적인 오류 감지 및 복구 메커니즘에 대한 기술적 세부 사항 부족

◦

다른 GPU 인프라 관리 시스템과의 성능 비교 정보 부재

◦

다양한 LLM 모델 및 훈련 작업에 대한 일반화 가능성 추가 검증 필요

◦

20만 개 이상의 GPU 환경에 대한 접근성 제한

Made with Slashpage