Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

A Theoretical Framework for Environmental Similarity and Vessel Mobility as Coupled Predictors of Marine Invasive Species Pathways

CareMedEval dataset: Evaluating Critical Appraisal and Reasoning in the Biomedical Field

Node-Based Editing for Multimodal Generation of Text, Audio, Image, and Video

Control Barrier Function for Aligning Large Language Models

A Criminology of Machines

FATE: A Formal Benchmark Series for Frontier Algebra of Multiple Difficulty Levels

PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contracts

Causal Graph Neural Networks for Healthcare

LA-MARRVEL: A Knowledge-Grounded and Language-Aware LLM Reranker for AI-MARRVEL in Rare Disease Diagnosis

OceanAI: A Conversational Platform for Accurate, Transparent, Near-Real-Time Oceanographic Insights

Pay for The Second-Best Service: A Game-Theoretic Approach Against Dishonest LLM Providers

Artificial Intelligence in Elementary STEM Education: A Systematic Review of Current Applications and Future Challenges

Non-Convex Over-the-Air Heterogeneous Federated Learning: A Bias-Variance Trade-off

TraceTrans: Translation and Spatial Tracing for Surgical Prediction

TowerVision: Understanding and Improving Multilinguality in Vision-Language Models

Gestura: A LVLM-Powered System Bridging Motion and Semantics for Real-Time Free-Form Gesture Understanding

ADPO: Anchored Direct Preference Optimization

Rewarding the Journey, Not Just the Destination: A Composite Path and Answer Self-Scoring Reward Mechanism for Test-Time Reinforcement Learning

RealDPO: Real or Not Real, that is the Preference

Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning

Mathematics with large language models as provers and verifiers

Integrating Sequential and Relational Modeling for User Events: Datasets and Prediction Tasks

A Systematic Evaluation of Self-Supervised Learning for Label-Efficient Sleep Staging with Wearable EEG

Learning to Navigate Socially Through Proactive Risk Perception

Training Large Language Models To Reason In Parallel With Global Forking Tokens

ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models

HyperAdapt: Simple High-Rank Adaptation

Comparing Computational Pathology Foundation Models using Representational Similarity Analysis

Back to Ear: Perceptually Driven High Fidelity Music Reconstruction

Test-Time Warmup for Multimodal Large Language Models

Zero-Shot Referring Expression Comprehension via Vison-Language True/False Verification

Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications

Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions

A Multi-target Bayesian Transformer Framework for Predicting Cardiovascular Disease Biomarkers during Pandemics

Med-GLIP: Advancing Medical Language-Image Pre-training with Large-scale Grounded Dataset

NyayaRAG: Realistic Legal Judgment Prediction with RAG under the Indian Common Law System

XRoboToolkit: A Cross-Platform Framework for Robot Teleoperation

Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agenda

GENIAL: Generative Design Space Exploration via Network Inversion for Low Power Algorithmic Logic Units

MeAJOR Corpus: A Multi-Source Dataset for Phishing Email Detection

Deep Graph Learning for Industrial Carbon Emission Analysis and Policy Impact

Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocation

Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization

HoliSafe: Holistic Safety Benchmarking and Modeling for Vision-Language Model

Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence

HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts

Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models

Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data

Hierarchical Retrieval with Evidence Curation for Open-Domain Financial Question Answering on Standardized Documents

Two Causally Related Needles in a Video Haystack

Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics

Autocomp: A Powerful and Portable Code Optimizer for Tensor Accelerators

But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectors

Learning Dynamics of RNNs in Closed-Loop Environments

Revisiting Residual Connections: Orthogonal Updates for Stable and Efficient Deep Networks

Multimodal Cancer Modeling in the Age of Foundation Model Embeddings

Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards

TathyaNyaya and FactLegalLlama: Advancing Factual Judgment Prediction and Explanation in the Indian Legal Context

Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models

Efficient Model Development through Fine-tuning Transfer

SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning

Pragmatic Reasoning improves LLM Code Generation

KGGen: Extracting Knowledge Graphs from Plain Text with Language Models

How Memory in Optimization Algorithms Implicitly Modifies the Loss

GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs

Transferable & Stealthy Ensemble Attacks: A Black-Box Jailbreaking Framework for Large Language Models

Understanding Adam Requires Better Rotation Dependent Assumptions

Beyond the Kolmogorov Barrier: A Learnable Weighted Hybrid Autoencoder for Model Order Reduction

Residual Kolmogorov-Arnold Network for Enhanced Deep Learning

Legal Fact Prediction: The Missing Piece in Legal Judgment Prediction

LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users

Projection Methods for Operator Learning and Universal Approximation

Stochastic Diffusion: A Diffusion Probabilistic Model for Stochastic Time Series Forecasting

A Unified Kernel for Neural Network Learning

Toward Autonomous Engineering Design: A Knowledge-Guided Multi-Agent Framework

SnapStream: Efficient Long Sequence Decoding on Dataflow Accelerators

Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learning

BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning

Toward Clinically Grounded Foundation Models in Pathology

Seg the HAB: Language-Guided Geospatial Algae Bloom Reasoning and Segmentation

A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning

Structured Debate Improves Corporate Credit Reasoning in Financial AI

Cross-modal Causal Intervention for Alzheimer's Disease Prediction

Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth

Style2Code: A Style-Controllable Code Generation Framework with Dual-Modal Contrastive Representation Learning

Building Altruistic and Moral AI Agent with Brain-inspired Emotional Empathy Mechanisms

"Let's Agree to Disagree": Investigating the Disagreement Problem in Explainable AI for Text Summarization

Collaboration Dynamics and Reliability Challenges of Multi-Agent LLM Systems in Finite Element Analysis

Discussion Graph Semantics of First-Order Logic with Equality for Reasoning about Discussion and Argumentation

X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations

Addressing divergent representations from causal interventions on neural networks

Integrating Temporal and Structural Context in Graph Transformers for Relational Deep Learning

LLM-as-a-Judge: Toward World Models for Slate Recommendation Systems

Are language models aware of the road not taken? Token-level uncertainty and hidden state dynamics

Alternative Fairness and Accuracy Optimization in Criminal Justice

RAGalyst: Automated Human-Aligned Agentic Evaluation for Domain-Specific RAG

Decoding Emergent Big Five Traits in Large Language Models: Temperature-Dependent Expression and Architectural Clustering

OUNLP at TSAR 2025 Shared Task: Multi-Round Text Simplifier via Code Generation

RUST-BENCH: Benchmarking LLM Reasoning on Unstructured Text within Structured Tables

Q3R: Quadratic Reweighted Rank Regularizer for Effective Low-Rank Training

PaperBench: Evaluating AI's Ability to Replicate AI Research

Created by

Haebom

저자

Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, Johannes Heidecke, Amelia Glaese, Tejal Patwardhan

개요

PaperBench는 최첨단 AI 연구를 복제하는 AI 에이전트의 능력을 평가하는 새로운 벤치마크입니다. 에이전트는 논문의 기여를 이해하고, 코드베이스를 개발하고, 실험을 성공적으로 실행하는 것을 포함하여 2024 ICML Spotlight 및 Oral 논문 20편을 처음부터 복제해야 합니다. 객관적인 평가를 위해 각 복제 작업을 명확한 채점 기준을 가진 더 작은 하위 작업으로 계층적으로 분해하는 등급 기준을 개발했습니다. PaperBench는 총 8,316개의 개별적으로 채점 가능한 작업으로 구성됩니다. 정확성과 현실성을 위해 각 ICML 논문의 저자와 함께 등급 기준을 공동 개발했습니다. 확장 가능한 평가를 가능하게 하기 위해 등급 기준에 따라 복제 시도를 자동으로 채점하는 LLM 기반 판정자를 개발하고 별도의 판정자용 벤치마크를 만들어 판정자의 성능을 평가했습니다. PaperBench에서 여러 최첨단 모델을 평가한 결과, 오픈 소스 스캐폴딩을 사용한 Claude 3.5 Sonnet (New)이라는 최고 성능의 에이전트가 평균 21.0%의 복제 점수를 달성했습니다. 마지막으로, 최고의 ML 박사 과정 학생들을 모집하여 PaperBench의 일부를 시도하게 하였고, 모델이 아직 인간 기준선을 능가하지 못함을 발견했습니다. 향후 AI 에이전트의 AI 엔지니어링 능력에 대한 이해를 촉진하기 위해 코드(https://github.com/openai/preparedness)를 오픈 소스로 공개했습니다.

시사점, 한계점

•

시사점:

◦

최첨단 AI 연구 복제 능력을 객관적으로 평가하는 새로운 벤치마크 (PaperBench) 제시.

◦

LLM 기반 자동 채점 시스템 개발을 통한 효율적인 평가 가능.

◦

현재 최고 수준의 AI 모델의 AI 연구 복제 능력 한계 제시 (평균 21% 성공률).

◦

오픈소스 공개를 통한 후속 연구 활성화.

•

한계점:

◦

현재 모델의 성능이 인간 전문가 수준에 미치지 못함.

◦

벤치마크에 포함된 논문의 수 제한 (20편).

◦

LLM 기반 판정자의 정확성에 대한 추가적인 검증 필요.

Made with Slashpage