Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

ACCeLLiuM: Supervised Fine-Tuning for Automated OpenACC Pragma Generation

AnchDrive: Bootstrapping Diffusion Policies with Hybrid Trajectory Anchors for End-to-End Driving

Diffusion-Augmented Contrastive Learning: A Noise-Robust Encoder for Biosignal Representations

FusedANN: Convexified Hybrid ANN via Attribute-Vector Fusion

HiCoLoRA: Addressing Context-Prompt Misalignment via Hierarchical Collaborative LoRA for Zero-Shot DST

A Longitudinal Randomized Control Study of Companion Chatbot Use: Anthropomorphism and Its Mediating Role on Social Impacts

TimeMosaic: Temporal Heterogeneity Guided Time Series Forecasting via Adaptive Granularity Patch and Segment-wise Decoding

Automated Facility Enumeration for Building Compliance Checking using Door Detection and Large Language Models

Dendritic Resonate-and-Fire Neuron for Effective and Efficient Long Sequence Modeling

Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on Math Textbook

RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation

Distribution-Aligned Decoding for Efficient LLM Task Adaptation

DivLogicEval: A Framework for Benchmarking Logical Reasoning Evaluation in Large Language Models

Recent Advancements in Microscopy Image Enhancement using Deep Learning: A Survey

Constructive Conflict-Driven Multi-Agent Reinforcement Learning for Strategic Diversity

Towards a Physics Foundation Model

Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews

Positional Encoding via Token-Aware Phase Attention

Chain or tree? Re-evaluating complex reasoning from the perspective of a matrix of thought

A Two-Stage Strategy for Mitosis Detection Using Improved YOLO11x Proposals and ConvNeXt Classification

JudgeAgent: Knowledge-wise and Dynamic LLM Evaluation with Agent-as-Interviewer

Draw-In-Mind: Rebalancing Designer-Painter Roles in Unified Multimodal Models Benefits Image Editing

Scalable Option Learning in High-Throughput Environments

"She was useful, but a bit too optimistic": Augmenting Design with Interactive Virtual Personas

In-Context Algorithm Emulation in Fixed-Weight Transformers

Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

Conflict-Aware Soft Prompting for Retrieval-Augmented Generation

ERIS: An Energy-Guided Feature Disentanglement Framework for Out-of-Distribution Time Series Classification

StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI

Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learning

Intuition emerges in Maximum Caliber models at criticality

GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy

Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models

Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning

SpectrumWorld: Artificial Intelligence Foundation for Spectroscopy

DAMR: Efficient and Adaptive Context-Aware Knowledge Graph Question Answering with LLM-Guided MCTS

Generative Logic: A New Computer Architecture for Deterministic Reasoning and Knowledge Generation

Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles

Hierarchical Graph Neural Network for Compressed Speech Steganalysis

R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning

The Invisible Leash: Why RLVR May or May Not Escape Its Origin

APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation

LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues

Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparities

KV Cache Steering for Controlling Frozen LLMs

Lightweight MSA Design Advances Protein Folding From Evolutionary Embeddings

Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning

Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions

Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoders

Neural-Network solver of ideal MHD equilibria

Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspective

Beyond Simple Graphs: Neural Multi-Objective Routing on Multigraphs

On the Necessity of Output Distribution Reweighting for Effective Class Unlearning

TAMMs: Temporal-Aware Multimodal Model for Satellite Image Change Understanding and Forecasting

Latent Concept Disentanglement in Transformer-based Language Models

Personalized LLM Decoding via Contrasting Personal Preference

Exploiting Block Coordinate Descent for Cost-Effective LLM Model Training

Security Degradation in Iterative AI Code Generation -- A Systematic Analysis of the Paradox

Think With Videos For Agentic Long-Video Understanding

VidBridge-R1: Bridging QA and Captioning for RL-based Video Understanding Models with Intermediate Proxy Tasks

Position: Simulating Society Requires Simulating Thought

AMPED: Adaptive Multi-objective Projection for balancing Exploration and skill Diversification

DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Models

Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement

Dual Branch VideoMamba with Gated Class Token Fusion for Violence Detection

CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech

Physics-Guided Motion Loss for Video Generation Model

Probing Neural Topology of Large Language Models

InfiMed: Low-Resource Medical MLLMs with Advancing Understanding and Reasoning

Mamba Integrated with Physics Principles Masters Long-term Chaotic System Forecasting

Model-Preserving Adaptive Rounding

DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation

SDPO: Importance-Sampled Direct Preference Optimization for Stable Diffusion Training

Spectral-inspired Operator Learning with Limited Data and Unknown Physics

BiomedSQL: Text-to-SQL for Scientific Reasoning on Biomedical Knowledge Bases

Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training

Prompting is not Enough: Exploring Knowledge Integration and Controllable Generation on Large Language Models

HD-PiSSA: High-Rank Distributed Orthogonal Adaptation

Can LLMs Alleviate Catastrophic Forgetting in Graph Continual Learning? A Systematic Study

FFT-based Dynamic Subspace Selection for Low-Rank Adaptive Optimization of Large Language Models

From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning

BP-Seg: A graphical model approach to unsupervised and non-contiguous text segmentation using belief propagation

Bottlenecked Transformers: Periodic KV Cache Consolidation for Generalised Reasoning

The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm

Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs

Learning Flexible Forward Trajectories for Masked Molecular Diffusion

Beyond Static Testbeds: An Interaction-Centric Agent Simulation Platform for Dynamic Recommender Systems

Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation

UniErase: Towards Balanced and Precise Unlearning in Language Models

Octic Vision Transformers: Quicker ViTs Through Equivariance

Intentional Gesture: Deliver Your Intentions with Gestures for Speech

UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models

VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluation

Learning Hierarchical Domain Models Through Environment-Grounded Interaction

Shadow-FT: Tuning Instruct Model via Training on Paired Base Model

Structured Relational Representations

Latent Veracity Inference for Identifying Errors in Stepwise Reasoning

Feature Hedging: Correlated Features Break Narrow Sparse Autoencoders

ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Training

Feature Hedging: Correlated Features Break Narrow Sparse Autoencoders

Created by

Haebom

저자

David Chanin, Toma\v{s} Dulka, Adria Garriga-Alonso

개요

희소 오토인코더(SAE)가 활성화 함수를 해석 가능한 선형 방향으로 분해한다고 가정하지만, underlying feature의 희소 선형 조합으로 구성되어 있을 경우에만 해당된다. 본 논문에서는 SAE가 훈련된 "진정한 feature"의 수보다 좁고, feature 간의 상관관계가 있는 경우, SAE가 상관관계가 있는 feature의 구성 요소를 병합하여 단일 의미성을 파괴한다는 것을 발견했다. 이러한 현상을 "feature hedging"이라고 하며, LLM SAE에서 이러한 두 가지 조건은 거의 확실하게 존재한다. Feature hedging은 SAE 재구성 손실에 의해 발생하며, SAE가 좁을수록 더 심각해진다. 본 연구에서는 feature hedging 문제를 소개하고, toy model에서 이론적으로, LLM에서 훈련된 SAE에서 경험적으로 연구한다. Feature hedging은 SAE가 supervised baseline보다 일관되게 성능이 떨어지는 주요 이유 중 하나일 수 있다고 추측한다. 마지막으로, feature hedging에 대한 이해를 바탕으로 matryoshka SAE의 개선된 변형을 제안한다. SAE 너비는 중립적인 하이퍼파라미터가 아니며, 좁은 SAE가 넓은 SAE보다 feature hedging의 영향을 더 많이 받는다는 것을 보여준다.

시사점, 한계점

•

시사점:

◦

SAE의 너비가 feature hedging에 영향을 미치는 중요한 하이퍼파라미터임을 밝힘. 좁은 SAE가 feature hedging에 더 취약함.

◦

LLM SAE에서 feature hedging의 존재 가능성을 제시하고, 성능 저하의 원인으로 지목함.

◦

feature hedging 현상에 대한 이해를 바탕으로 matryoshka SAE의 개선된 변형을 제안함.

•

한계점:

◦

Toy model 및 실험을 통해 feature hedging을 연구하였으나, 실제 LLM 환경에서의 영향력 및 개선 효과에 대한 추가 연구가 필요함.

◦

feature hedging이 SAE의 성능 저하를 유발하는 주요 원인 중 하나일 수 있다고 추측하지만, 다른 요인과의 정확한 관계에 대한 추가적인 분석이 필요함.

◦

제안된 matryoshka SAE의 개선된 변형의 성능에 대한 광범위한 검증이 필요함.

Made with Slashpage