Daily Arxiv

전 세계에서 발간되는 인공지능 관련 논문을 정리하는 페이지 입니다.
본 페이지는 Google Gemini를 활용해 요약 정리하며, 비영리로 운영 됩니다.
논문에 대한 저작권은 저자 및 해당 기관에 있으며, 공유 시 출처만 명기하면 됩니다.

VerilogLAVD: LLM-Aided Rule Generation for Vulnerability Detection in Verilog

HRS: Hybrid Representation Framework with Scheduling Awareness for Time Series Forecasting in Crowdsourced Cloud-Edge Platforms

Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward

Vehicle detection from GSV imagery: Predicting travel behaviour for cycling and motorcycling using Computer Vision

CRED-SQL: Enhancing Real-world Large Scale Database Text-to-SQL Parsing through Cluster Retrieval and Execution Description

Robust Federated Learning under Adversarial Attacks via Loss-Based Client Clustering

SRMA-Mamba: Spatial Reverse Mamba Attention Network for Pathological Liver Segmentation in MRI Volumes

MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graph

Fortifying the Agentic Web: A Unified Zero-Trust Architecture Against Logic-layer Threats

Generalized invariants meet constitutive neural networks: A novel framework for hyperelastic materials

What Matters for Bioacoustic Encoding

Age-Normalized HRV Features for Non-Invasive Glucose Prediction: A Pilot Sleep-Aware Machine Learning Study

Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamics

FreeGAD: A Training-Free yet Effective Approach for Graph Anomaly Detection

An Explainable AI based approach for Monitoring Animal Health

Exploring Content and Social Connections of Fake News with Explainable Text and Graph Learning

Can AI Keep a Secret? Contextual Integrity Verification: A Provable Security Architecture for LLMs

Fine-Grained Safety Neurons with Training-Free Continual Projection to Reduce LLM Fine Tuning Risks

To Theoretically Understand Transformer-Based In-Context Learning for Optimizing CSMA

MAViS: A Multi-Agent Framework for Long-Sequence Video Storytelling

Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression

Early Detection of Pancreatic Cancer Using Multimodal Learning on Electronic Health Records

A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges

FlowState: Sampling Rate Invariant Time Series Forecasting

Automatic Image Colorization with Convolutional Neural Networks and Generative Adversarial Networks

Unlocking the Potential of MLLMs in Referring Expression Segmentation via a Light-weight Mask Decoder

SlotMatch: Distilling Temporally Consistent Object-Centric Representations for Unsupervised Video Segmentation

SBP-YOLO:A Lightweight Real-Time Model for Detecting Speed Bumps and Potholes

Dataset Condensation with Color Compensation

A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model

Spatial-Temporal Transformer with Curriculum Learning for EEG-Based Emotion Recognition

RAPNet: A Receptive-Field Adaptive Convolutional Neural Network for Pansharpening

Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Data

Identify, Isolate, and Purge: Mitigating Hallucinations in LVLMs via Self-Evolving Distillation

Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs

Segment Anything in Pathology Images with Natural Language

Neural Cellular Automata for ARC-AGI

Scaling Intelligence: Designing Data Centers for Next-Gen Language Models

ConTextTab: A Semantics-Aware Tabular In-Context Learner

PlantDeBERTa: An Open Source Language Model for Plant Science

Recipes for Pre-training LLMs with MXFP8

Enhancing Visual Reliance in Text Generation: A Bayesian Perspective on Mitigating Hallucination in Large Vision-Language Models

G1: Teaching LLMs to Reason on Graphs with Reinforcement Learning

Sample Complexity of Diffusion Model Training Without Empirical Risk Minimizer Access

"Haet Bhasha aur Diskrimineshun": Phonetic Perturbations in Code-Mixed Hinglish to Red-Team LLMs

VoiceCloak: A Multi-Dimensional Defense Framework against Unauthorized Diffusion-based Voice Cloning

LaDi-WM: A Latent Diffusion-based World Model for Predictive Manipulation

Position: We Need Responsible, Application-Driven (RAD) AI Research

Blending 3D Geometry and Machine Learning for Multi-View Stereopsis

Harnessing Structured Knowledge: A Concept Map-Based Approach for High-Quality Multiple Choice Question Generation with Effective Distractors

Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models

POPri: Private Federated Learning using Preference-Optimized Synthetic Data

Parameter-Efficient Continual Fine-Tuning: A Survey

Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling

Augmented Adversarial Trigger Learning

Setup Once, Secure Always: A Single-Setup Secure Federated Learning Aggregation Protocol with Forward and Backward Secrecy for Dynamic Users

DDD-GenDT: Dynamic Data-driven Generative Digital Twin Framework

Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes

Script-Strategy Aligned Generation: Aligning LLMs with Expert-Crafted Dialogue Scripts and Therapeutic Strategies for Psychotherapy

SSD-TS: Exploring the Potential of Linear State Space Models for Diffusion Models in Time Series Imputation

Vision Backbone Efficient Selection for Image Classification in Low-Data Regimes

Boolean Matrix Logic Programming on the GPU

Iterative Utility Judgment Framework via LLMs Inspired by Relevance in Philosophy

iTBLS: A Dataset of Interactive Conversations Over Tabular Information

Fusing Echocardiography Images and Medical Records for Continuous Patient Stratification

Joint Problems in Learning Multiple Dynamical Systems

Radio Map Estimation: Empirical Validation and Analysis

"I see models being a whole other thing": An Empirical Study of Pre-Trained Model Naming Conventions and A Tool for Enhancing Naming Consistency

LEGO: Learning and Graph-Optimized Modular Tracker for Online Multi-Object Tracking with Point Clouds

Adaptation and Optimization of Automatic Speech Recognition (ASR) for the Maritime Domain in the Field of VHF Communication

PC-Sampler: Position-Aware Calibration of Decoding Bias in Masked Diffusion Models

Modeling Relational Logic Circuits for And-Inverter Graph Convolutional Network

FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction

Scaling Up without Fading Out: Goal-Aware Sparse GNN for RL-based Generalized Planning

DualSG: A Dual-Stream Explicit Semantic-Guided Multivariate Time Series Forecasting Framework

Modeling Uncertainty: Constraint-Based Belief States in Imperfect-Information Games

Data-Efficient Safe Policy Improvement Using Parametric Structure

Towards Urban Planing AI Agent in the Age of Agentic AI

Dispositions and Roles of Generically Dependent Entities

Efficient Network Automatic Relevance Determination

Modeling the Diachronic Evolution of Legal Norms: An LRMoo-Based, Component-Level Approach

Language-Guided Multi-Agent Learning in Simulations: A Unified Framework and Evaluation

It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics

Hierarchical Reinforcement Learning in Multi-Goal Spatial Navigation with Autonomous Mobile Robots

Trust, but verify

Two Heads are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoning

Hawkeye:Efficient Reasoning with Model Collaboration

GoAI: Enhancing AI Students' Learning Paths and Idea Generation via Graph of AI Ideas

The StudyChat Dataset: Student Dialogues With ChatGPT in an Artificial Intelligence Course

VRoPE: Rotary Position Embedding for Video Large Language Models

Where to Go Next Day: Multi-scale Spatial-Temporal Decoupled Model for Mid-term Human Mobility Prediction

LEGO-GraphRAG: Modularizing Graph-based Retrieval-Augmented Generation for Design Space Exploration

GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation

Unintended Misalignment from Agentic Fine-Tuning: Risks and Mitigation

Ask Good Questions for Large Language Models

Efficient Knowledge Graph Unlearning with Zeroth-order Information

Evaluating Identity Leakage in Speaker De-Identification Systems

ASDFormer: A Transformer with Mixtures of Pooling-Classifier Experts for Robust Autism Diagnosis and Biomarker Discovery

Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation

Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization

Recipes for Pre-training LLMs with MXFP8

Created by

Haebom

저자

Asit Mishra, Dusan Stosic, Simon Layton, Paulius Micikevicius

개요

본 논문은 NVIDIA Blackwell 세대 GPU에서 도입된 Microscaling(MX) 포맷을 사용하여 모델 파라미터 및 관련 텐서를 더 적은 비트로 표현하는 기법을 다룹니다. MX 포맷은 좁은 부동 소수점 데이터 타입과 더 세분화된 블록별 스케일링 계수를 결합하여 기존 방식보다 더 많은 텐서의 양자화와 효율적인 연산 실행을 가능하게 합니다. 논문에서는 MX 포맷의 효과적인 사용을 위한 다양한 매개변수 선택에 대해 검토하고, MXFP8-E4M3 데이터 타입과 특정 변환 알고리즘을 사용하여 BF16을 사용한 학습 결과와 동일한 결과를 얻는 방법을 제시합니다. 최대 80억 개의 파라미터를 가진 모델을 최대 15조 토큰의 고품질 데이터셋으로 학습한 결과를 보여줍니다.

시사점, 한계점

•

시사점:

◦

MX 포맷을 이용한 효율적인 모델 학습 방법 제시

◦

BF16과 동일한 성능을 달성하는 MXFP8-E4M3 데이터 타입 및 변환 알고리즘 제안

◦

대규모 모델(최대 80억 파라미터) 및 대규모 데이터셋(최대 15조 토큰)에서의 실험 결과 제시

◦

GPU 메모리 효율성 향상 및 학습 속도 개선 가능성 제시

•

한계점:

◦

제시된 방법이 모든 모델과 데이터셋에 적용 가능한지는 추가 연구 필요

◦

특정 NVIDIA GPU 아키텍처에 종속적인 결과일 가능성 존재

◦

다른 양자화 기법과의 비교 분석 부족

Made with Slashpage