Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Driving

Swin-TUNA : A Novel PEFT Approach for Accurate Food Image Segmentation

EarthLink: A Self-Evolving AI Agent for Climate Science

Reality Proxy: Fluid Interactions with Real-World Objects in MR via Abstract Representations

Leveraging multi-source and heterogeneous signals for fatigue detection

Segmentation-free Goodness of Pronunciation

Adaptive Relative Pose Estimation Framework with Dual Noise Tuning for Safe Approaching Maneuvers

Compositional Coordination for Multi-Robot Teams with Large Language Models

Diffusion Beats Autoregressive in Data-Constrained Settings

The New LLM Bottleneck: A Systems Perspective on Latent Attention and Mixture-of-Experts

EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Control

Long-Short Distance Graph Neural Networks and Improved Curriculum Learning for Emotion Recognition in Conversation

Omni-Thinker: Scaling Cross-Domain Generalization in LLMs via Multi-Task RL with Hybrid Rewards

GCC-Spam: Spam Detection via GAN, Contrastive Learning, and Character Similarity Networks

SDSC:A Structure-Aware Metric for Semantic Signal Representation Learning

Multilingual LLMs Are Not Multilingual Thinkers: Evidence from Hindi Analogy Evaluation

Frequency-Dynamic Attention Modulation for Dense Prediction

A Survey of Deep Learning for Geometry Problem Solving

EEG Foundation Models: A Critical Review of Current Progress and Future Directions

Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models

A PBN-RL-XAI Framework for Discovering a "Hit-and-Run" Therapeutic Strategy in Melanoma

Task Priors: Enhancing Model Evaluation by Considering the Entire Space of Downstream Tasks

OrQstrator: An AI-Powered Framework for Advanced Quantum Circuit Optimization

A comprehensive study of LLM-based argument classification: from LLAMA through GPT-4o to Deepseek-R1

Mechanistic Indicators of Understanding in Large Language Models

Scaling RL to Long Videos

Fast Bilateral Teleoperation and Imitation Learning Using Sensorless Force Control via Accurate Dynamics Model

Masked Autoencoders that Feel the Heart: Unveiling Simplicity Bias for ECG Analyses

SyncMapV2: Robust and Adaptive Unsupervised Segmentation

LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs

Why Do Class-Dependent Evaluation Effects Occur with Time Series Feature Attributions? A Synthetic Data Investigation

Diffuse and Disperse: Image Generation with Representation Regularization

LLM-D12: A Dual-Dimensional Scale of Instrumental and Relational Dependencies on Large Language Models

MambaNeXt-YOLO: A Hybrid State Space Model for Real-time Object Detection

PALADIN : Robust Neural Fingerprinting for Text-to-Image Diffusion Models

Outcome-Based Online Reinforcement Learning: Algorithms and Fundamental Limits

Machine Learning Solutions Integrated in an IoT Healthcare Platform for Heart Failure Risk Stratification

Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning

Vision Transformers in Precision Agriculture: A Comprehensive Survey

PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding

Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research

LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important

Trigger without Trace: Towards Stealthy Backdoor Attack on Text-to-Image Diffusion Models

Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs

Aligning Vision to Language: Annotation-Free Multimodal Knowledge Graph Construction for Enhanced LLMs Reasoning

Att-Adapter: A Robust and Precise Domain-Specific Multi-Attributes T2I Diffusion Adapter via Conditional Variational Autoencoder

When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning

Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation

Tackling Hallucination from Conditional Models for Medical Image Reconstruction with DynamicDPS

Quantum Machine Learning in Precision Medicine and Drug Discovery -- A Game Changer for Tailored Treatments?

A general language model for peptide identification

ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models

EVEv2: Improved Baselines for Encoder-Free Vision-Language Models

LLM Alignment as Retriever Optimization: An Information Retrieval Perspective

Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settings

Online Housing Market

Integrated Learning and Optimization for Congestion Management and Profit Maximization in Real-Time Electricity Market

Integrating Evidence into the Design of XAI and AI-based Decision Support Systems: A Means-End Framework for End-users in Construction

Scalable Parameter Design for Superconducting Quantum Circuits with Graph Neural Networks

A Survey of Event Causality Identification: Taxonomy, Challenges, Assessment, and Prospects

Neural Corrective Machine Unranking

Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation

Differentiable Motion Manifold Primitives for Reactive Motion Generation under Kinodynamic Constraints

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces

RUMI: Rummaging Using Mutual Information

Neural Machine Unranking

VolDoGer: LLM-assisted Datasets for Domain Generalization in Vision-Language Tasks

Unsupervised Concept Drift Detection from Deep Learning Representations in Real-time

A Multi-Faceted Evaluation Framework for Assessing Synthetic Data Generated by Large Language Models

DualXDA: Towards Sparse, Efficient and Explainable Data Attribution in Large AI Models

Quantifying the Uniqueness and Divisiveness of Presidential Discourse

DocTER: Evaluating Document-based Knowledge Editing

Learning Concepts Definable in First-Order Logic with Counting

Recognizing and Eliciting Weakly Single Crossing Profiles on Trees

Compliance Brain Assistant: Conversational Agentic AI for Assisting Compliance Tasks in Enterprise Environments

Learning Temporal Abstractions via Variational Homomorphisms in Option-Induced Abstract MDPs

When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systems

An Integrated Framework of Prompt Engineering and Multidimensional Knowledge Graphs for Legal Dispute Analysis

DisMS-TS: Eliminating Redundant Multi-Scale Features for Time Series Classification

Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games

Beamforming and Resource Allocation for Delay Minimization in RIS-Assisted OFDM Systems

Neurodivergent Influenceability as a Contingent Solution to the AI Alignment Problem

EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework

SuperARC: An Agnostic Test for Narrow, General, and Super Intelligence Based On the Principles of Recursive Compression and Algorithmic Probability

IPCGRL: Language-Instructed Reinforcement Learning for Procedural Level Generation

OR-LLM-Agent: Automating Modeling and Solving of Operations Research Optimization Problems with Reasoning LLM

Chemical reasoning in LLMs unlocks strategy-aware synthesis planning and reaction mechanism elucidation

BEARCUBS: A benchmark for computer-using web agents

From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems

HPS: Hard Preference Sampling for Human Preference Alignment

A Differentiated Reward Method for Reinforcement Learning based Multi-Vehicle Cooperative Decision-Making Algorithms

Retrieving Classes of Causal Orders with Inconsistent Knowledge Bases

On the Structure of Game Provenance and its Applications

I-CEE: Tailoring Explanations of Image Classification Models to User Expertise

SIDA: Synthetic Image Driven Zero-shot Domain Adaptation

3D Software Synthesis Guided by Constraint-Expressive Intermediate Representation

Moving Out: Physically-grounded Human-AI Collaboration

SynC: Synthetic Image Caption Dataset Refinement with One-to-many Mapping for Zero-shot Image Captioning

Approximate SMT Counting Beyond Discrete Domains

DRWKV: Focusing on Object Edges for Low-Light Image Enhancement

Zeroth-Order Fine-Tuning of LLMs in Random Subspaces

Created by

Haebom

저자

Ziming Yu, Pan Zhou, Sike Wang, Jia Li, Mi Tian, Hua Huang

개요

본 논문은 대규모 언어 모델(LLM)의 미세 조정을 위한 메모리 효율적인 최적화 방법인 SubZero를 제안합니다. 기존의 영차(Zeroth-order) 최적화 방법들은 기울기 추정의 분산이 모델의 차원에 따라 선형적으로 증가하는 문제점을 가지고 있는데, SubZero는 저차원 섭동을 이용하여 이 문제를 해결합니다. SubZero는 메모리 소비를 줄이면서 훈련 성능을 향상시키며, 기존의 영차 최적화 방법들보다 빠른 수렴 속도를 보입니다. 실험 결과를 통해 다양한 언어 모델링 작업에서 SubZero의 우수성을 검증하고, 소스 코드를 공개하였습니다.

시사점, 한계점

•

시사점:

◦

대규모 언어 모델 미세 조정을 위한 메모리 효율적인 새로운 최적화 기법 제시

◦

기존 영차 최적화 방법들의 한계점인 높은 기울기 추정 분산 문제 해결

◦

향상된 훈련 성능 및 빠른 수렴 속도 달성

◦

실제 언어 모델링 작업에서의 효과 검증 및 소스 코드 공개

•

한계점:

◦

제시된 방법의 일반적인 LLM 구조 및 크기 이외의 다른 구조나 크기에 대한 일반화 가능성에 대한 추가 연구 필요

◦

다양한 하이퍼파라미터 튜닝 및 최적화에 대한 추가 연구 필요

◦

실험 결과가 특정 데이터셋 및 작업에 국한될 수 있으므로, 더욱 광범위한 실험이 필요함.

Made with Slashpage