Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

A Systematic Review of Human-AI Co-Creativity

DFVEdit: Conditional Delta Flow Vector for Zero-shot Video Editing

Exploring the Capabilities of the Frontier Large Language Models for Nuclear Energy Research

MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering

Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity

FEAST: A Flexible Mealtime-Assistance System Towards In-the-Wild Personalization

How do Probabilistic Graphical Models and Graph Neural Networks Look at Network Data?

Vision Transformers Don't Need Trained Registers

Eye of Judgement: Dissecting the Evaluation of Russian-speaking LLMs with POLLUX

Maximizing Confidence Alone Improves Reasoning

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models

Improving LLM Outputs Against Jailbreak Attacks with Expert Model Integration

Personalized Robotic Object Rearrangement from Scene Context

Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs

OpenTCM: A GraphRAG-Empowered LLM-based System for Traditional Chinese Medicine Knowledge Retrieval and Diagnosis

Explicit neural network classifiers for non-separable data

USM-VC: Mitigating Timbre Leakage with Universal Semantic Mapping Residual Block for Voice Conversion

Towards Adaptive Memory-Based Optimization for Enhanced Retrieval-Augmented Generation

LoopGen: Training-Free Loopable Music Generation

Automated detection of atomicity violations in large-scale systems

Grammar and Gameplay-aligned RL for Game Description Generation with LLMs

Generative AI for Software Architecture. Applications, Challenges, and Future Directions

English K_Quantization of LLMs Does Not Disproportionately Diminish Multilingual Performance

Heuristics for AI-driven Graphical Asset Generation Tools in Game Design and Development Pipelines: A User-Centred Approach

Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truth

Multi-Turn Code Generation Through Single-Step Rewards

Round Attention: A Novel Round-Level Attention Mechanism to Accelerate LLM Inference

KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and Document Understanding

AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers

MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot

Generative Data Mining with Longtail-Guided Diffusion

Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation

No More Sliding Window: Efficient 3D Medical Image Segmentation with Differentiable Top-k Patch Sampling

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

End-to-End Long Document Summarization using Gradient Caching

Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models

KNN-MMD: Cross Domain Wireless Sensing via Local Distribution Alignment

EUR/USD Exchange Rate Forecasting incorporating Text Mining Based on Pre-trained Language Models and Deep Learning Methods

Large-Scale Multirobot Coverage Path Planning on Grids With Path Deconfliction

Dynamic Adaptive Rank Space Exploration for Efficient Sentiment Analysis with Large Language Models

Federated Data-Efficient Instruction Tuning for Large Language Models

QT-DoG: Quantization-aware Training for Domain Generalization

Testing Causal Models with Hidden Variables in Polynomial Delay via Conditional Independencies

Stability of Primal-Dual Gradient Flow Dynamics for Multi-Block Convex Optimization Problems

LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagation

The Mamba in the Llama: Distilling and Accelerating Hybrid Models

EUR-USD Exchange Rate Forecasting Based on Information Fusion with Large Language Models and Deep Learning Methods

Dynamic Adaptive Optimization for Effective Sentiment Analysis Fine-Tuning on Large Language Models

Mitigating Metropolitan Carbon Emissions with Dynamic Eco-driving at Scale

CAPM: Fast and Robust Verification on Maxpool-based CNN via Dual Network

MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance

From Data Quality for AI to AI for Data Quality: A Systematic Review of Tools for AI-Augmented Data Quality Management in Data Warehouses

FuzzAug: Data Augmentation by Coverage-guided Fuzzing for Neural Test Generation

RLSF: Fine-tuning LLMs via Symbolic Feedback

A Survey on Patent Analysis: From NLP to Multimodal AI

Enhancing Object Detection Robustness: Detecting and Restoring Confidence in the Presence of Adversarial Patch Attacks

Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought

Programming Distributed Collective Processes in the eXchange Calculus

Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentum

Fairness and Bias in Algorithmic Hiring: a Multidisciplinary Survey

On CNF formulas irredundant with respect to unit clause propagation

SONG: Self-Organizing Neural Graphs

Mobile-R1: Towards Interactive Reinforcement Learning for VLM-Based Mobile Agent via Task-Level Rewards

KunLunBaizeRAG: Reinforcement Learning Driven Inference Performance Leap for Large Language Models

FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoring

Dynamic Knowledge Exchange and Dual-diversity Review: Concisely Unleashing the Potential of a Multi-Agent Research Team

PhysUniBench: An Undergraduate-Level Physics Reasoning Benchmark for Multimodal Models

From Human to Machine Psychology: A Conceptual Framework for Understanding Well-Being in Large Language Models

The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason

VLM@school -- Evaluation of AI image understanding on German middle school knowledge

The AI Imperative: Scaling High-Quality Peer Review in Machine Learning

Toward Data Systems That Are Business Semantic Centric and AI Agents Assisted

ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows

$C^3$-Bench: The Things Real Disturbing LLM based Agent in Multi-Tasking

StarFT: Robust Fine-tuning of Zero-shot Models via Spuriosity Alignment

REMOR: Automated Peer Review Generation with LLM Reasoning and Multi-Objective Reinforcement Learning

Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'

Local Markov Equivalence and Local Causal Discovery for Identifying Controlled Direct Effects

Adapting Probabilistic Risk Assessment for AI

From Superficial to Deep: Integrating External Knowledge for Follow-up Question Generation Using Knowledge Graph and LLM

Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search

SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models

OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis

Problem Solving Through Human-AI Preference-Based Cooperation

CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents

CLoVE: Personalized Federated Learning through Clustering of Loss Vector Embeddings

HyperCLOVA X THINK Technical Report

Dehazing Light Microscopy Images with Guided Conditional Flow Matching: finding a sweet spot between fidelity and realism

QuickSilver -- Speeding up LLM Inference through Dynamic Token Halting, KV Skipping, Contextual Token Fusion, and Adaptive Matryoshka Quantization

Multi-View Contrastive Learning for Robust Domain Adaptation in Medical Time Series Analysis

Towards Distributed Neural Architectures

Can Video Large Multimodal Models Think Like Doubters-or Double-Down: A Study on Defeasible Video Entailment

Probabilistic Optimality for Inference-time Scaling

Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems

From Ground to Air: Noise Robustness in Vision Transformers and CNNs for Event-Based Vehicle Classification with Potential UAV Applications

Concept-Level AI for Telecom: Moving Beyond Large Language Models

A Framework for Multi-source Privacy Preserving Epidemic Analysis

A Deep Learning framework for building damage assessment using VHR SAR and geospatial data: demonstration on the 2023 Turkiye Earthquake

Less Greedy Equivalence Search

A Practical Approach to Power Saving in Hearables Using Sub-Nyquist Sampling with Bandwidth Extension

LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation

작성자

Haebom

저자

Juzheng Zhang, Jiacheng You, Ashwinee Panda, Tom Goldstein

개요

LoRA는 매개변수 효율적인 미세 조정(PEFT) 방법으로 널리 사용되지만, 여전히 상당한 오버헤드가 발생하고 다중 작업 시나리오에서 매개변수 간섭 문제가 발생합니다. 본 논문에서는 간단하면서도 효과적인 접근 방식인 LoRA with Reduced Interference (LoRI)를 제안합니다. LoRI는 투영 행렬 A를 무작위 투영으로 고정하고, B 행렬을 작업별 마스크를 사용하여 희소화합니다. 이 설계는 훈련 가능한 매개변수의 수를 크게 줄이면서 강력한 작업 성능을 유지합니다. 또한, LoRI는 어댑터 하위 공간 간의 직교성을 활용하여 어댑터 병합에서 작업 간 간섭을 최소화하고, 희소성을 사용하여 파국적인 망각을 완화하여 지속적인 학습을 지원합니다. 자연어 이해, 수학적 추론, 코드 생성 및 안전 정렬 작업에 대한 광범위한 실험을 통해 LoRI가 LoRA보다 최대 95% 적은 훈련 가능한 매개변수를 사용하면서 전체 미세 조정 및 기존 PEFT 방법보다 우수한 성능을 보임을 보여줍니다. 다중 작업 실험에서 LoRI는 작업 간 간섭을 줄이고 효과적인 어댑터 병합 및 지속적인 학습을 가능하게 합니다. 코드는 https://github.com/juzhengz/LoRI 에서 확인할 수 있습니다.

GitHub - juzhengz/LoRI

Contribute to juzhengz/LoRI development by creating an account on GitHub.

시사점, 한계점

•

시사점:

◦

LoRA의 한계점인 오버헤드 및 다중 작업 시나리오에서의 매개변수 간섭 문제를 효과적으로 해결합니다.

◦

LoRA보다 최대 95% 적은 훈련 가능한 매개변수를 사용하면서 우수한 성능을 달성합니다.

◦

다중 작업 시나리오에서 효과적인 어댑터 병합 및 지속적인 학습을 지원합니다.

◦

자연어 이해, 수학적 추론, 코드 생성, 안전 정렬 등 다양한 작업에서 우수한 성능을 입증했습니다.

•

한계점:

◦

본 논문에서 제시된 LoRI의 한계점에 대한 명시적인 언급이 없습니다. 추가적인 실험이나 분석을 통해 다양한 상황에서의 일반화 성능이나 특정 작업에 대한 취약점 등이 추가적으로 검토될 필요가 있습니다.

◦

무작위 투영과 희소화 전략의 최적화 방안에 대한 추가적인 연구가 필요할 수 있습니다.

Slashpage로 제작됨