Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

On the optimization dynamics of RLVR: Gradient gap and step size thresholds

Synthetic Series-Symbol Data Generation for Time Series Foundation Models

Learning Neural Exposure Fields for View Synthesis

Fewer Weights, More Problems: A Practical Attack on LLM Pruning

Can Lessons From Human Teams Be Applied to Multi-Agent Systems? The Role of Structure, Diversity, and Interaction Dynamics

Haystack Engineering: Context Engineering for Heterogeneous and Agentic Long-Context Evaluation

GyroSwin: 5D Surrogates for Gyrokinetic Plasma Turbulence Simulations

Online Rubrics Elicitation from Pairwise Comparisons

Scalable multilingual PII annotation for responsible AI in LLMs

Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods

Reinforce-Ada: An Adaptive Sampling Framework for Reinforce-Style LLM Training

Revealing Interconnections between Diseases: from Statistical Methods to Large Language Models

P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs

MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering

Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention

Spatiotemporal Forecasting as Planning: A Model-Based Reinforcement Learning Approach with Generative World Models

6G-Enabled Digital Twin Framework for Real-Time Cyber-Physical Systems: An Experimental Validation with Industrial Bearing Fault Detection

InstructPLM-mu: 1-Hour Fine-Tuning of ESM2 Beats ESM3 in Protein Mutation Predictions

Prompt-Aware Scheduling for Low-Latency LLM Serving

Learning Robust Diffusion Models from Imprecise Supervision

CLARITY: Clinical Assistant for Routing, Inference, and Triage

Nav-EE: Navigation-Guided Early Exiting for Efficient Vision-Language Models in Autonomous Driving

Machine Learning for Detection and Analysis of Novel LLM Jailbreaks

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

TimeScope: Towards Task-Oriented Temporal Grounding In Long Videos

Editable Noise Map Inversion: Encoding Target-image into Noise For High-Fidelity Image Manipulation

ClustRecNet: A Novel End-to-End Deep Learning Framework for Clustering Algorithm Recommendation

Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithm

InfiR2: A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models

Benchmarking and Mitigate Sycophancy in Medical Vision-Language Models

CFDLLMBench: A Benchmark Suite for Evaluating Large Language Models in Computational Fluid Dynamics

A Mega-Study of Digital Twins Reveals Strengths, Weaknesses and Opportunities for Further Improvement

Robust LLM Training Infrastructure at ByteDance

RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation

MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment

Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal

Individual utilities of life satisfaction reveal inequality aversion unrelated to political alignment

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens

From Federated Learning to X-Learning: Breaking the Barriers of Decentrality Through Random Walks

Latent Variable Modeling in Multi-Agent Reinforcement Learning via Expectation-Maximization for UAV-Based Wildlife Protection

Quantifying Label-Induced Bias in Large Language Model Self- and Cross-Evaluations

Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices

On Task Vectors and Gradients

On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting

LATTE: Learning Aligned Transactions and Textual Embeddings for Bank Clients

ACD-CLIP: Decoupling Representation and Dynamic Fusion for Zero-Shot Anomaly Detection

Neural Beam Field for Spatial Beam RSRP Prediction

AMFT: Aligning LLM Reasoners by Meta-Learning the Optimal Imitation-Exploration Balance

Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling

VAGPO: Vision-augmented Asymmetric Group Preference Optimization for Graph Routing Problems

TriP-LLM: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detection

Enabling Self-Improving Agents to Learn at Test Time With Human-In-The-Loop Guidance

Preprint: Poster: Did I Just Browse A Website Written by LLMs?

Site-Level Fine-Tuning with Progressive Layer Freezing: Towards Robust Prediction of Bronchopulmonary Dysplasia from Day-1 Chest Radiographs in Extremely Preterm Infants

AirScape: An Aerial Generative World Model with Motion Controllability

Tuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training

EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework

Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments with a Hierarchical Spatial-Cognition Long-Short Memory System

Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspective

Interpretable and Granular Video-Based Quantification of Motor Characteristics from the Finger Tapping Test in Parkinson Disease

Bures-Wasserstein Flow Matching for Graph Generation

Symmetry in Neural Network Parameter Spaces

CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models

Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs

AD-EE: Early Exiting for Fast and Reliable Vision-Language Models in Autonomous Driving

Robustness in Both Domains: CLIP Needs a Robust Text Encoder

Elicit and Enhance: Advancing Multimodal Reasoning in Medical Scenarios

Any-to-Bokeh: Arbitrary-Subject Video Refocusing with Video Diffusion Model

An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks

FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information

Beyond Demonstrations: Dynamic Vector Construction from Latent Representations

DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaboration

Sequential Monte Carlo for Policy Optimization in Continuous POMDPs

Game of Trust: How Trustworthy Does Your Blockchain Think You Are?

Collaborative Unlabeled Data Optimization

Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Features

System Prompt Optimization with Meta-Learning

Cognitio Emergens: Agency, Dimensions, and Dynamics in Human-AI Knowledge Co-Creation

Multimodal Language Models See Better When They Look Shallower

Exploring human-SAV interaction using LLMs: The impact of psychological factors on user experience

On Developers' Self-Declaration of AI-Generated Code: An Analysis of Practices

Diffusion Generative Recommendation with Continuous Tokens

TARO: Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning for Synchronized Video-to-Audio Synthesis

DeepOHeat-v1: Efficient Operator Learning for Fast and Trustworthy Thermal Simulation and Optimization in 3D-IC Design

Brain2Text Decoding Model Reveals the Neural Mechanisms of Visual Semantic Processing

Issue Localization via LLM-Driven Iterative Code Graph Searching

CCDP: Composition of Conditional Diffusion Policies with Guided Sampling

Measuring directional bias amplification in image captions using predictability

Contrastive Learning Augmented Social Recommendations

WyckoffDiff -- A Generative Diffusion Model for Crystal Symmetry

Solving Linear-Gaussian Bayesian Inverse Problems with Decoupled Diffusion Sequential Monte Carlo

RadVLM: A Multitask Conversational Vision-Language Model for Radiology

IG-MCTS: Human-in-the-Loop Cooperative Navigation under Incomplete Information

OrcaLoca: An LLM Agent Framework for Software Issue Localization

Enabling Population-Level Parallelism in Tree-Based Genetic Programming for GPU Acceleration

AD-LLM: Benchmarking Large Language Models for Anomaly Detection

Preference Discerning with LLM-Enhanced Generative Retrieval

SwarmGPT: Combining Large Language Models with Safe Motion Planning for Drone Swarm Choreography

Generate Any Scene: Scene Graph Driven Data Synthesis for Visual Generation Training

Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens

Created by

Haebom

저자

Eugene Kwek, Wenpeng Yin

개요

본 논문은 대규모 언어 모델(LLM)의 메모리, 지연 시간 및 서비스 비용 효율성을 개선하기 위해 제안된 COMPACT라는 새로운 프루닝(pruning) 기법을 소개합니다. COMPACT는 희귀 단어 임베딩/LM 헤드 레이어를 축소하고, 공통 토큰 가중 활성화를 사용하여 FFN 중간 채널을 프루닝하여 LLM과 소형 언어 모델(SLM) 모두에서 성능을 유지하면서 표준 변환기(transformer) 아키텍처를 유지합니다. 실험 결과, Qwen, LLaMA, Gemma 계열 모델(0.5B-70B)에서 매개변수, GPU 메모리, 지연 시간을 대폭 줄이면서 최첨단 성능을 달성했습니다.

시사점, 한계점

•

시사점:

◦

표준 변환기 아키텍처를 유지하여 배포 용이성을 확보했습니다.

◦

어휘 및 FFN 프루닝 사이의 균형을 통해 규모에 적응 가능합니다.

◦

경쟁력 있는 프루닝 시간과 함께 강력한 메모리 절감 효과를 제공합니다.

◦

다양한 모델 크기(0.5B-70B)에서 우수한 성능을 보입니다.

•

한계점:

◦

본 논문의 한계점에 대한 직접적인 언급은 요약문에 포함되어 있지 않습니다. (논문 전체를 읽어봐야 함)

Made with Slashpage