Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles

Saturation Self-Organizing Map

PiPViT: Patch-based Visual Interpretable Prototypes for Retinal Image Analysis

Specification and Evaluation of Multi-Agent LLM Systems -- Prototype and Cybersecurity Applications

Intra-Trajectory Consistency for Reward Modeling

Foundation Models in Medical Imaging -- A Review and Outlook

Merging Smarter, Generalizing Better: Enhancing Model Merging on OOD Data

Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spaces

Improving Large Language Models with Concept-Aware Fine-Tuning

Consistent Video Editing as Flow-Driven Image-to-Video Generation

Evidential Spectrum-Aware Contrastive Learning for OOD Detection in Dynamic Graphs

Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning

DURA-CPS: A Multi-Role Orchestrator for Dependability Assurance in LLM-Enabled Cyber-Physical Systems

Cartridges: Lightweight and general-purpose long context representations via self-study

Fast-DataShapley: Neural Modeling for Training Data Valuation

NOBLE -- Neural Operator with Biologically-informed Latent Embeddings to Capture Experimental Variability in Biological Neuron Models

LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation

Can AI Master Econometrics? Evidence from Econometrics AI Agent on Expert-Level Tasks

Improving the Calibration of Confidence Scores in Text Generation Using the Output Distribution's Characteristics

Training RL Agents for Multi-Objective Network Defense Tasks

Explainability of Large Language Models using SMILE: Statistical Model-agnostic Interpretability with Local Explanations

RSCF: Relation-Semantics Consistent Filter for Entity Embedding of Knowledge Graph

MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE

Impact of Frame Rates on Speech Tokenizer: A Case Study on Mandarin and English

Can reasoning models comprehend mathematical problems in Chinese ancient texts? An empirical study based on data from Suanjing Shishu

Table-R1: Region-based Reinforcement Learning for Table Understanding

Spatiotemporal Field Generation Based on Hybrid Mamba-Transformer with Physics-informed Fine-tuning

Graph-Based Floor Separation Using Node Embeddings and Clustering of WiFi Trajectories

Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Exploration

Quantitative Analysis of Performance Drop in DeepSeek Model Quantization

PhysNav-DG: A Novel Adaptive Framework for Robust VLM-Sensor Fusion in Navigation Applications

FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents

ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness

"It's not a representation of me": Examining Accent Bias and Digital Exclusion in Synthetic AI Voice Services

Towards Personalized Conversational Sales Agents: Contextual User Profiling for Strategic Action

PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices

Beyond the Visible: Multispectral Vision-Language Learning for Earth Observation

Ensemble Knowledge Distillation for Machine Learning Interatomic Potentials

Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts

V-Max: A Reinforcement Learning Framework for Autonomous Driving

PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts

Naturalistic Computational Cognitive Science: Towards generalizable models and theories that capture the full range of natural behavior

MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis

The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training

Understanding the Emergence of Multimodal Representation Alignment

Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis

TUMLU: A Unified and Native Language Understanding Benchmark for Turkic Languages

BalanceBenchmark: A Survey for Multimodal Imbalance Learning

AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers

Vision-Language Models for Edge Networks: A Comprehensive Survey

Foundation Models for Anomaly Detection: Vision and Challenges

Unsafe LLM-Based Search: Quantitative Analysis and Mitigation of Safety Risks in AI Web Search

Capturing Temporal Dynamics in Large-Scale Canopy Tree Height Estimation

Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderation

Evaluating Sample Utility for Efficient Data Selection by Mimicking Model Weights

Discovering Hidden Visual Concepts Beyond Linguistic Input in Infant Learning

Bi-directional Mapping of Morphology Metrics and 3D City Blocks for Enhanced Characterization and Generation of Urban Form

One Diffusion to Generate Them All

Entropy Controllable Direct Preference Optimization

TrajAgent: An LLM-based Agent Framework for Automated Trajectory Modeling via Collaboration of Large and Small Models

Control Industrial Automation System with Large Language Model Agents

How Well Do Large Language Models Serve as End-to-End Secure Code Agents for Python?

Self-interpreting Adversarial Images

Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective

Ad Auctions for LLMs via Retrieval Augmented Generation

Dynamic and Adaptive Feature Generation with LLM

MiniMaxAD: A Lightweight Autoencoder for Feature-Rich Anomaly Detection

PLeak: Prompt Leaking Attacks against Large Language Model Applications

HandS3C: 3D Hand Mesh Reconstruction with State Space Spatial Channel Attention from RGB images

DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learning

Evolution Guided Generative Flow Networks

Manipulating Feature Visualizations with Gradient Slingshots

Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning

Agent Semantics, Semantic Spacetime, and Graphical Reasoning

Decomposability-Guaranteed Cooperative Coevolution for Large-Scale Itinerary Planning

DeePoly: A High-Order Accuracy Scientific Machine Learning Framework for Function Approximation and Solving PDEs

Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models

The Automated but Risky Game: Modeling Agent-to-Agent Negotiations and Transactions in Consumer Markets

ChemRxivQuest: A Curated Chemistry Question-Answer Database Extracted from ChemRxiv Preprints

Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly `Know When They Do Not Know'

Enhancing multimodal analogical reasoning with Logic Augmented Generation

Beating Transformers using Synthetic Cognition

HypRL: Reinforcement Learning of Control Policies for Hyperproperties

From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Paper

Bel Esprit: Multi-Agent Framework for Building AI Model Pipelines

Dynamic Policy Fusion for User Alignment Without Re-Interaction

Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing

An overview of domain-specific foundation model: key technologies, applications and challenges

Learning from Litigation: Graphs and LLMs for Retrieval and Reasoning in eDiscovery

Combining Deep Reinforcement Learning and Search with Generative Models for Game-Theoretic Opponent Modeling

EMLoC: Emulator-based Memory-efficient Fine-tuning with LoRA Correction

code_transformed: The Influence of Large Language Models on Code

Reimagining Dance: Real-time Music Co-creation between Dancers and AI

Upgrade or Switch: Do We Need a New Registry Architecture for the Internet of AI Agents?

VGR: Visual Grounded Reasoning

Technical Evaluation of a Disruptive Approach in Homomorphic AI

SAIL: Faster-than-Demonstration Execution of Imitation Learning Policies

Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe?

Today's Cat Is Tomorrow's Dog: Accounting for Time-Based Changes in the Labels of ML Vulnerability Detection Approaches

Emergent Specialization: Rare Token Neurons in Language Models

Created by

Haebom

저자

Jing Liu, Haozheng Wang, Yueheng Li

개요

본 논문은 대규모 언어 모델이 특수 분야에서 중요한 희귀 토큰을 표현하고 생성하는 데 어려움을 겪는다는 문제를 다룹니다. 연구에서는 희귀 토큰 예측에 특별히 강력한 영향을 미치는 뉴런 구조, 즉 '희귀 토큰 뉴런'을 식별하고, 이 뉴런의 출현 및 동작 메커니즘을 조사합니다. 이러한 뉴런은 훈련 중 동적으로 등장하는 특징적인 3단계 구조(플래토, 거듭제곱 법칙, 급격한 감소)를 보이며, 균질한 초기 상태에서 기능적으로 차별화된 구조로 진화합니다. 활성화 공간에서 희귀 토큰 뉴런은 다른 뉴런과의 공동 활성화를 피하면서 선택적으로 공동 활성화되는 조정된 하위 네트워크를 형성합니다. 이러한 기능적 특수화는 무거운 꼬리를 가진 가중치 분포의 발달과 상관관계가 있을 가능성이 있으며, 등장하는 특수화에 대한 통계 역학적 기반을 제시합니다.

시사점, 한계점

•

시사점:

◦

희귀 토큰 처리에 특화된 뉴런 구조의 존재를 밝힘으로써 대규모 언어 모델의 내부 동작에 대한 이해를 심화시킴.

◦

희귀 토큰 뉴런의 3단계 구조 및 기능적 특수화 메커니즘에 대한 통찰력 제공.

◦

희귀 토큰 처리 성능 향상을 위한 새로운 알고리즘 및 모델 설계 방향 제시 가능성.

◦

통계 역학적 관점에서 희귀 토큰 처리 문제에 대한 새로운 접근 방식 제시.

•

한계점:

◦

현재는 관찰된 현상에 대한 상관관계를 제시하였으나, 인과관계에 대한 추가적인 연구가 필요함.

◦

특정 모델 및 데이터셋에 국한된 결과일 가능성 존재. 다양한 모델 및 데이터셋에 대한 추가 실험 필요.

◦

희귀 토큰 뉴런의 기능적 특수화가 실제 성능 향상으로 직결되는지에 대한 추가적인 분석 필요.

◦

희귀 토큰 뉴런의 발견이 모든 유형의 희귀 토큰에 적용 가능한지에 대한 추가 검증 필요.

Made with Slashpage