Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 1

T2I-Copilot: A Training-Free Multi-Agent Text-to-Image System for Enhanced Prompt Interpretation and Interactive Generation

The Xeno Sutra: Can Meaning and Value be Ascribed to an AI-Generated "Sacred" Text?

Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots

Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering

The Carbon Cost of Conversation, Sustainability in the Age of Language Models

Machine Learning Risk Intelligence for Green Hydrogen Investment: Insights for Duqm R3 Auction

Differential-UMamba: Rethinking Tumor Segmentation Under Limited Data Scenarios

Decoding Instructional Dialogue: Human-AI Collaborative Analysis of Teacher Use of AI Tool at Scale

Towards Facilitated Fairness Assessment of AI-based Skin Lesion Classifiers Through GenAI-based Image Synthesis

Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Models

Machine learning-based multimodal prognostic models integrating pathology images and high-throughput omic data for overall survival prediction in cancer: a systematic review

Diffusion Beats Autoregressive in Data-Constrained Settings

SegQuant: A Semantics-Aware and Generalizable Quantization Framework for Diffusion Models

Linguistic and Embedding-Based Profiling of Texts generated by Humans and Large Language Models

AI-ming backwards: Vanishing archaeological landscapes in Mesopotamia and automatic detection of sites on CORONA imagery

Kodezi Chronos: A Debugging-First Language Model for Repository-Scale Code Understanding

TolerantECG: A Foundation Model for Imperfect Electrocardiogram

AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model

Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models

HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation

ZERO: Industry-ready Vision Foundation Model with Multi-modal Prompts

End-to-End Large Portfolio Optimization for Variance Minimization with Neural Networks through Covariance Cleaning

iPanda: An LLM-based Agent for Automated Conformance Testing of Communication Protocols

SurgiSR4K: A High-Resolution Endoscopic Video Dataset for Robotic-Assisted Minimally Invasive Procedures

Generating Heterogeneous Multi-dimensional Data : A Comparative Study

Automated Generation of Diverse Courses of Actions for Multi-Agent Operations using Binary Optimization and Graph Learning

GLIMPSE: Holistic Cross-Modal Explainability for Large Vision-Language Models

Fine-Grained Perturbation Guidance via Attention Head Selection

FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluation

SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs

LinkTo-Anime: A 2D Animation Optical Flow Dataset from 3D Model Rendering

Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models

Adversarial bandit optimization for approximately linear functions

FB-RAG: Improving RAG with Forward and Backward Lookup

Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Sampling

Learning Pareto-Optimal Rewards from Noisy Preferences: A Framework for Multi-Objective Inverse Reinforcement Learning

Context-Aware Probabilistic Modeling with LLM for Multimodal Time Series Forecasting

Multi-Modal Hypergraph Enhanced LLM Learning for Recommendation

OPAL: Encoding Causal Understanding of Physical Systems for Robot Learning

Not someone, but something: Rethinking trust in the age of medical AI

My Life in Artificial Intelligence: People, anecdotes, and some lessons learnt

SQuat: Subspace-orthogonal KV Cache Quantization

Levels of Analysis for Large Language Models

Narrative Context Protocol: An Open-Source Storytelling Framework for Generative AI

Conceptualizing Uncertainty: A Concept-based Approach to Explaining Uncertainty

Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning

A calibration test for evaluating set-based epistemic uncertainty representations

Multi-branch of Attention Yields Accurate Results for Tabular Data

PAR-AdvGAN: Improving Adversarial Attack Capability with Progressive Auto-Regression AdvGAN

Latent Swap Joint Diffusion for 2D Long-Form Latent Generation

LIMO: Less is More for Reasoning

Implementing Large Quantum Boltzmann Machines as Generative AI Models for Dataset Balancing

A Survey on Memory-Efficient Transformer-Based Model Training in AI for Science

Breaking Memory Limits: Gradient Wavelet Transform Enhances LLMs Training

Back Home: A Computer Vision Solution to Seashell Identification for Ecological Restoration

DIVE: Taming DINO for Subject-Driven Video Editing

AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning

Puzzle Similarity: A Perceptually-guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructions

Receding Hamiltonian-Informed Optimal Neural Control and State Estimation for Closed-Loop Dynamical Systems

Fuse Before Transfer: Knowledge Fusion for Heterogeneous Distillation

Recovering Manifold Structure Using Ollivier-Ricci Curvature

Simulated patient systems are intelligent when powered by large language model-based AI agents

GSON: A Group-based Social Navigation Framework with Large Multimodal Model

Task Arithmetic for Language Expansion in Speech Translation

Geometric Algebra Meets Large Language Models: Instruction-Based Transformations of Separate Meshes in 3D, Interactive and Controllable Scenes

Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs

IRASim: A Fine-Grained World Model for Robot Manipulation

MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Data

Long-Term Fairness Inquiries and Pursuits in Machine Learning: A Survey of Notions, Methods, and Challenges

Online Concurrent Multi-Robot Coverage Path Planning

The pitfalls of next-token prediction

Adversarial attacks and defenses in explainable artificial intelligence: A survey

Demystifying Misconceptions in Social Bots Research

A Multi-Agent System Enables Versatile Information Extraction from the Chemical Literature

What Does 'Human-Centred AI' Mean?

Illuminating the Three Dogmas of Reinforcement Learning under Evolutionary Light

Enhancing Student Learning with LLM-Generated Retrieval Practice Questions: An Empirical Study in Data Science Courses

SLR: Automated Synthesis for Scalable Logical Reasoning

Towards Reliable Proof Generation with LLMs: A Neuro-Symbolic Approach

2D-Curri-DPO: Two-Dimensional Curriculum Learning for Direct Preference Optimization

An Algebraic Approach to Moralisation and Triangulation of Probabilistic Graphical Models

SAKE: Steering Activations for Knowledge Editing

Intrinsic Barriers and Practical Pathways for Human-AI Alignment: An Agreement-Based Complexity Analysis

A Scalable Approach to Probabilistic Neuro-Symbolic Robustness Verification

Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data

Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search

A finite time analysis of distributed Q-learning

Foundation Models for Demand Forecasting via Dual-Strategy Ensembling

Supervised Quantum Image Processing

Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security

ReXGroundingCT: A 3D Chest CT Dataset for Segmentation of Findings from Free-Text Reports

XAI for Point Cloud Data using Perturbations based on Meaningful Segmentation

Exploring the Stratified Space Structure of an RL Game with the Volume Growth Transform

Bridging Synthetic and Real-World Domains: A Human-in-the-Loop Weakly-Supervised Framework for Industrial Toxic Emission Segmentation

Staining and locking computer vision models without retraining

Teach Me to Trick: Exploring Adversarial Transferability via Knowledge Distillation

ChemDFM-R: An Chemical Reasoner LLM Enhanced with Atomized Chemical Knowledge

Fine-Tuning Code Language Models to Detect Cross-Language Bugs

MapAgent: Trajectory-Constructed Memory-Augmented Planning for Mobile Task Automation

The pitfalls of next-token prediction

Created by

Haebom

저자

Gregor Bachmann, Vaishnavh Nagarajan

개요

본 논문은 단순한 다음 토큰 예측 모델이 인간의 지능을 충실하게 모델링할 수 있는지에 대한 의문을 제기한다. 다음 토큰 예측의 두 가지 측면인 자동 회귀 추론과 강제 교사 학습을 구분하여, 강제 교사 학습이 정확한 다음 토큰 예측기를 학습하지 못하는 경우가 있음을 주장한다. 특히, 특정 유형의 작업에서는 강제 교사 학습이 정확한 다음 토큰 예측기를 학습하지 못할 수 있으며, 그 메커니즘을 설명하고 Transformer와 Mamba 아키텍처가 간단한 계획 작업에서도 실패하는 실험적 증거를 제시한다. 마지막으로, 더미 토큰을 사용하여 여러 토큰을 미리 예측하는 '교사 없는' 학습을 통해 이러한 실패를 해결할 수 있는 초기 증거를 제공하며, 다음 토큰 예측 패러다임을 넘어서는 연구를 촉구한다.

시사점, 한계점

•

시사점: 다음 토큰 예측 모델의 한계를 명확히 제시하고, 강제 교사 학습의 실패 가능성과 그 메커니즘을 밝힘으로써, 인공지능 모델의 한계를 극복하기 위한 새로운 연구 방향을 제시한다. '교사 없는' 학습이라는 대안적인 학습 방법의 가능성을 보여준다.

•

한계점: '교사 없는' 학습의 효과는 초기 단계의 증거에 그치며, 더욱 폭넓은 실험과 분석이 필요하다. 제시된 계획 작업은 특정 유형의 작업에 국한되어 있으며, 다른 유형의 작업에도 적용 가능한지에 대한 추가 연구가 필요하다.

Made with Slashpage