Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 1

T2I-Copilot: A Training-Free Multi-Agent Text-to-Image System for Enhanced Prompt Interpretation and Interactive Generation

The Xeno Sutra: Can Meaning and Value be Ascribed to an AI-Generated "Sacred" Text?

Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots

Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering

The Carbon Cost of Conversation, Sustainability in the Age of Language Models

Machine Learning Risk Intelligence for Green Hydrogen Investment: Insights for Duqm R3 Auction

Differential-UMamba: Rethinking Tumor Segmentation Under Limited Data Scenarios

Decoding Instructional Dialogue: Human-AI Collaborative Analysis of Teacher Use of AI Tool at Scale

Towards Facilitated Fairness Assessment of AI-based Skin Lesion Classifiers Through GenAI-based Image Synthesis

Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Models

Machine learning-based multimodal prognostic models integrating pathology images and high-throughput omic data for overall survival prediction in cancer: a systematic review

Diffusion Beats Autoregressive in Data-Constrained Settings

SegQuant: A Semantics-Aware and Generalizable Quantization Framework for Diffusion Models

Linguistic and Embedding-Based Profiling of Texts generated by Humans and Large Language Models

AI-ming backwards: Vanishing archaeological landscapes in Mesopotamia and automatic detection of sites on CORONA imagery

Kodezi Chronos: A Debugging-First Language Model for Repository-Scale Code Understanding

TolerantECG: A Foundation Model for Imperfect Electrocardiogram

AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model

Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models

HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation

ZERO: Industry-ready Vision Foundation Model with Multi-modal Prompts

End-to-End Large Portfolio Optimization for Variance Minimization with Neural Networks through Covariance Cleaning

iPanda: An LLM-based Agent for Automated Conformance Testing of Communication Protocols

SurgiSR4K: A High-Resolution Endoscopic Video Dataset for Robotic-Assisted Minimally Invasive Procedures

Generating Heterogeneous Multi-dimensional Data : A Comparative Study

Automated Generation of Diverse Courses of Actions for Multi-Agent Operations using Binary Optimization and Graph Learning

GLIMPSE: Holistic Cross-Modal Explainability for Large Vision-Language Models

Fine-Grained Perturbation Guidance via Attention Head Selection

FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluation

SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs

LinkTo-Anime: A 2D Animation Optical Flow Dataset from 3D Model Rendering

Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models

Adversarial bandit optimization for approximately linear functions

FB-RAG: Improving RAG with Forward and Backward Lookup

Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Sampling

Learning Pareto-Optimal Rewards from Noisy Preferences: A Framework for Multi-Objective Inverse Reinforcement Learning

Context-Aware Probabilistic Modeling with LLM for Multimodal Time Series Forecasting

Multi-Modal Hypergraph Enhanced LLM Learning for Recommendation

OPAL: Encoding Causal Understanding of Physical Systems for Robot Learning

Not someone, but something: Rethinking trust in the age of medical AI

My Life in Artificial Intelligence: People, anecdotes, and some lessons learnt

SQuat: Subspace-orthogonal KV Cache Quantization

Levels of Analysis for Large Language Models

Narrative Context Protocol: An Open-Source Storytelling Framework for Generative AI

Conceptualizing Uncertainty: A Concept-based Approach to Explaining Uncertainty

Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning

A calibration test for evaluating set-based epistemic uncertainty representations

Multi-branch of Attention Yields Accurate Results for Tabular Data

PAR-AdvGAN: Improving Adversarial Attack Capability with Progressive Auto-Regression AdvGAN

Latent Swap Joint Diffusion for 2D Long-Form Latent Generation

LIMO: Less is More for Reasoning

Implementing Large Quantum Boltzmann Machines as Generative AI Models for Dataset Balancing

A Survey on Memory-Efficient Transformer-Based Model Training in AI for Science

Breaking Memory Limits: Gradient Wavelet Transform Enhances LLMs Training

Back Home: A Computer Vision Solution to Seashell Identification for Ecological Restoration

DIVE: Taming DINO for Subject-Driven Video Editing

AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning

Puzzle Similarity: A Perceptually-guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructions

Receding Hamiltonian-Informed Optimal Neural Control and State Estimation for Closed-Loop Dynamical Systems

Fuse Before Transfer: Knowledge Fusion for Heterogeneous Distillation

Recovering Manifold Structure Using Ollivier-Ricci Curvature

Simulated patient systems are intelligent when powered by large language model-based AI agents

GSON: A Group-based Social Navigation Framework with Large Multimodal Model

Task Arithmetic for Language Expansion in Speech Translation

Geometric Algebra Meets Large Language Models: Instruction-Based Transformations of Separate Meshes in 3D, Interactive and Controllable Scenes

Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs

IRASim: A Fine-Grained World Model for Robot Manipulation

MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Data

Long-Term Fairness Inquiries and Pursuits in Machine Learning: A Survey of Notions, Methods, and Challenges

Online Concurrent Multi-Robot Coverage Path Planning

The pitfalls of next-token prediction

Adversarial attacks and defenses in explainable artificial intelligence: A survey

Demystifying Misconceptions in Social Bots Research

A Multi-Agent System Enables Versatile Information Extraction from the Chemical Literature

What Does 'Human-Centred AI' Mean?

Illuminating the Three Dogmas of Reinforcement Learning under Evolutionary Light

Enhancing Student Learning with LLM-Generated Retrieval Practice Questions: An Empirical Study in Data Science Courses

SLR: Automated Synthesis for Scalable Logical Reasoning

Towards Reliable Proof Generation with LLMs: A Neuro-Symbolic Approach

2D-Curri-DPO: Two-Dimensional Curriculum Learning for Direct Preference Optimization

An Algebraic Approach to Moralisation and Triangulation of Probabilistic Graphical Models

SAKE: Steering Activations for Knowledge Editing

Intrinsic Barriers and Practical Pathways for Human-AI Alignment: An Agreement-Based Complexity Analysis

A Scalable Approach to Probabilistic Neuro-Symbolic Robustness Verification

Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data

Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search

A finite time analysis of distributed Q-learning

Foundation Models for Demand Forecasting via Dual-Strategy Ensembling

Supervised Quantum Image Processing

Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security

ReXGroundingCT: A 3D Chest CT Dataset for Segmentation of Findings from Free-Text Reports

XAI for Point Cloud Data using Perturbations based on Meaningful Segmentation

Exploring the Stratified Space Structure of an RL Game with the Volume Growth Transform

Bridging Synthetic and Real-World Domains: A Human-in-the-Loop Weakly-Supervised Framework for Industrial Toxic Emission Segmentation

Staining and locking computer vision models without retraining

Teach Me to Trick: Exploring Adversarial Transferability via Knowledge Distillation

ChemDFM-R: An Chemical Reasoner LLM Enhanced with Atomized Chemical Knowledge

Fine-Tuning Code Language Models to Detect Cross-Language Bugs

MapAgent: Trajectory-Constructed Memory-Augmented Planning for Mobile Task Automation

Breaking Memory Limits: Gradient Wavelet Transform Enhances LLMs Training

Created by

Haebom

저자

Ziqing Wen, Ping Luo, Jiahuan Wang, Xiaoge Deng, Jinping Zou, Kun Yuan, Tao Sun, Dongsheng Li

개요

본 논문은 대규모 언어 모델(LLM) 학습 시 메모리 효율적인 최적화를 위한 새로운 방법인 Gradient Wavelet Transform (GWT)를 제안합니다. 기존의 메모리 효율적인 알고리즘들은 특이값 분해 투영이나 가중치 고정과 같은 방법에 의존하여 성능 저하를 야기하지만, GWT는 웨이블릿 변환을 기울기(gradient)에 적용하여 메모리 요구량을 크게 줄이는 동시에 성능 저하 없이 효율적인 학습을 가능하게 합니다. 전처리 및 미세 조정 작업에 대한 광범위한 실험을 통해 GWT가 메모리 사용량과 학습 성능 측면에서 기존의 고급 메모리 효율적인 최적화 기법 및 전랭크 접근 방식보다 최첨단 성능을 달성함을 보여줍니다.

시사점, 한계점

•

시사점:

◦

대규모 언어 모델 학습 시 메모리 제약을 극복하는 새로운 효율적인 방법 제시

◦

기존의 저랭크 훈련 방식보다 우수한 성능과 메모리 효율성 달성

◦

GWT는 Adam과 같은 메모리 집약적인 최적화 기법과 원활하게 통합 가능

◦

전처리 및 미세 조정 작업 모두에서 우수한 성능 검증

•

한계점:

◦

GWT의 일반화 성능에 대한 추가적인 연구 필요

◦

다양한 LLM 아키텍처 및 데이터셋에 대한 추가적인 실험 필요

◦

GWT의 계산 복잡도 및 구현의 어려움에 대한 평가 필요

Made with Slashpage