Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

Distillation Robustifies Unlearning

Cartridges: Lightweight and general-purpose long context representations via self-study

Text-to-LoRA: Instant Transformer Adaption

Information Bargaining: Bilateral Commitment in Bayesian Persuasion

Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models

Heartcare Suite: Multi-dimensional Understanding of ECG with Raw Multi-lead Signal Modeling

Peer-Ranked Precision: Creating a Foundational Dataset for Fine-Tuning Vision Models from DataSeeds' Annotated Imagery

TissUnet: Improved Extracranial Tissue and Cranium Segmentation for Children through Adulthood

A Red Teaming Roadmap Towards System-Level Safety

Reason-to-Recommend: Using Interaction-of-Thought Reasoning to Enhance LLM Recommendation

Context Is Not Comprehension

Feature-Based Lie Group Transformer for Real-World Applications

Horizon Reduction Makes RL Scalable

SLAC: Simulation-Pretrained Latent Action Space for Whole-Body Real-World RL

A Diffusion-Driven Temporal Super-Resolution and Spatial Consistency Enhancement Framework for 4D MRI imaging

BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidance

Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds

Deep Learning for Retinal Degeneration Assessment: A Comprehensive Analysis of the MARIO AMD Progression Challenge

CoT is Not True Reasoning, It Is Just a Tight Constraint to Imitate: A Theory Perspective

Rethinking the effects of data contamination in Code Intelligence

MINT: Multimodal Instruction Tuning with Multimodal Interaction Grouping

Protap: A Benchmark for Protein Modeling on Realistic Downstream Applications

NTPP: Generative Speech Language Modeling for Dual-Channel Spoken Dialogue via Next-Token-Pair Prediction

SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions

Learning from Double Positive and Unlabeled Data for Potential-Customer Identification

Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Race

Diversity of Transformer Layers: One Aspect of Parameter Scaling Laws

Noise-Robustness Through Noise: Asymmetric LoRA Adaption with Poisoning Expert

Large Language Models Often Know When They Are Being Evaluated

WorkForceAgent-R1: Incentivizing Reasoning Capability in LLM-based Web Agents via Reinforcement Learning

CAST: Contrastive Adaptation and Distillation for Semi-Supervised Instance Segmentation

PartInstruct: Part-level Instruction Following for Fine-grained Robot Manipulation

RainFusion: Adaptive Video Generation Acceleration via Multi-Dimensional Visual Redundancy

VeriThoughts: Enabling Automated Verilog Code Generation using Reasoning and Formal Verification

Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles

APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation

When Two LLMs Debate, Both Think They'll Win

Turb-L1: Achieving Long-term Turbulence Tracing By Tackling Spectral Bias

GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains

Can MLLMs Guide Me Home? A Benchmark Study on Fine-Grained Visual Reasoning from Transit Maps

Sample Complexity of Diffusion Model Training Without Empirical Risk Minimizer Access

EVADE: Multimodal Benchmark for Evasive Content Detection in E-Commerce Applications

Simulating Macroeconomic Expectations using LLM Agents

Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models

Mechanistic evaluation of Transformers and state space models

Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models

Pel, A Programming Language for Orchestrating AI Agents

MARVEL: Multi-Agent RTL Vulnerability Extraction using Large Language Models

Learning Pareto-Optimal Rewards from Noisy Preferences: A Framework for Multi-Objective Inverse Reinforcement Learning

Q-Policy: Quantum-Enhanced Policy Evaluation for Scalable Reinforcement Learning

BLEUBERI: BLEU is a surprisingly effective reward for instruction following

Position: We Need Responsible, Application-Driven (RAD) AI Research

Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approach

LookAlike: Consistent Distractor Generation in Math MCQs

Tree-Sliced Wasserstein Distance with Nonlinear Projection

Test-time Correlation Alignment

Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics

OpenTCM: A GraphRAG-Empowered LLM-based System for Traditional Chinese Medicine Knowledge Retrieval and Diagnosis

A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment

MIB: A Mechanistic Interpretability Benchmark

Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation

LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models

Toward Total Recall: Enhancing FAIRness through AI-Driven Metadata Standardization

Finding Interest Needle in Popularity Haystack: Improving Retrieval by Modeling Item Exposure

Geometrical Properties of Text Token Embeddings for Strong Semantic Binding in Text-to-Image Generation

sudo rm -rf agentic_security

Imagine to Hear: Auditory Knowledge Generation can be an Effective Assistant for Language Models

Towards Achieving Perfect Multimodal Alignment

nvBench 2.0: Resolving Ambiguity in Text-to-Visualization through Stepwise Reasoning

FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA

RONA: Pragmatically Diverse Image Captioning with Coherence Relations

Unifying 2D and 3D Vision-Language Understanding

Revisiting semi-supervised learning in the era of foundation models

AI-based Framework for Robust Model-Based Connector Mating in Robotic Wire Harness Installation

Generalized Interpolating Discrete Diffusion

Towards Autonomous Reinforcement Learning for Real-World Robotic Manipulation with Large Language Models

Straight-Line Diffusion Model for Efficient 3D Molecular Generation

Examining the Mental Health Impact of Misinformation on Social Media Using a Hybrid Transformer-Based Approach

Dynamic spillovers and investment strategies across artificial intelligence ETFs, artificial intelligence tokens, and green markets

Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMs

LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement

from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors

EgoNormia: Benchmarking Physical Social Norm Understanding

PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation

NeoBERT: A Next-Generation BERT

From Offline to Online Memory-Free and Task-Free Continual Learning via Fine-Grained Hypergradients

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection

SYNTHIA: Novel Concept Design with Affordance Composition

DBudgetKV: Dynamic Budget in KV Cache Compression for Ensuring Optimal Performance

AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay

DISC: DISC: Dynamic Decomposition Improves LLM Inference Scaling

Predicting Bad Goods Risk Scores with ARIMA Time Series: A Novel Risk Assessment Approach

Space-O-RAN: Enabling Intelligent, Open, and Interoperable Non Terrestrial Networks in 6G

MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-Text Decoding

Improving the Diffusability of Autoencoders

Tree-of-Debate: Multi-Persona Debate Trees Elicit Critical Thinking for Scientific Comparative Analysis

MMTEB: Massive Multilingual Text Embedding Benchmark

From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MARKERGEN

How Expressive are Knowledge Graph Foundation Models?

Machine Learning Should Maximize Welfare, but Not by (Only) Maximizing Accuracy

Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics

Created by

Haebom

저자

Cong Xu, Wenbin Liang, Mo Yu, Anan Liu, Ke-Yue Zhang, Shunli Wang, Lizhuang Ma, Jianyong Wang, Jun Wang, Wei Zhang

개요

본 논문은 모델의 급격한 확장으로 인해 높아진 훈련 및 미세 조정 비용 문제를 해결하기 위해, 메모리 소비의 주요 원인인 상태 유지 최적화기(예: Adam)의 메모리 사용량을 줄이는 새로운 최적화기 SOLO를 제시합니다. SOLO는 기존의 8비트 또는 4비트 양자화 기반 방법보다 더 나아가, Adam 스타일 최적화기를 3비트 또는 심지어 2비트의 낮은 정밀도로 양자화된 상태를 유지하도록 합니다. 이는 부호 없는 양자화에서 발생하는 신호 압도 문제와 부호 있는 양자화에서 증가하는 기울기 분산 문제를 해결하기 위한 이론적 분석과 맞춤형 로그 양자화 및 정밀도 특정 모멘텀 하이퍼파라미터 사용을 통해 가능해졌습니다. 결과적으로 SOLO는 Adam 스타일 최적화기에 손쉽게 적용되어 최소한의 정확도 손실로 상당한 메모리 절약을 제공합니다.

시사점, 한계점

•

시사점:

◦

Adam 스타일 최적화기의 메모리 사용량을 획기적으로 줄이는 새로운 방법 제시.

◦

3비트 또는 2비트의 낮은 정밀도 양자화에서도 효과적인 최적화 가능성 증명.

◦

신호 압도 문제와 기울기 분산 문제에 대한 해결책 제시.

◦

최소한의 정확도 손실로 상당한 메모리 절약 달성.

•

한계점:

◦

논문에서 제시된 이론적 분석 및 해결책의 일반화 가능성에 대한 추가적인 검증 필요.

◦

다양한 모델 및 데이터셋에 대한 실험적 결과가 더 필요함.

◦

2비트 양자화의 실제 효율성 및 안정성에 대한 추가적인 연구가 필요함.

Made with Slashpage