Method: Evaluated robustness of watermarking techniques against paraphrasing and back translation attacks, and assessed quality and writing style preservation using linguistic metrics.

Result: Watermarking preserves semantics but deviates from original writing style and is susceptible to adversarial attacks, particularly back translation.

Conclusion: Current watermarking techniques need improvement to maintain text quality and resist adversarial attacks for wider adoption.

Abstract: To mitigate the potential harms of Large Language Models (LLMs)generated text, researchers have proposed watermarking, a process of embedding detectable signals within text. With watermarking, we can always accurately detect LLM-generated texts. However, recent findings suggest that these techniques often negatively affect the quality of the generated texts, and adversarial attacks can strip the watermarking signals, causing the texts to possibly evade detection. These findings have created resistance in the wide adoption of watermarking by LLM creators. Finally, to encourage adoption, we evaluate the robustness of several watermarking techniques to adversarial attacks by comparing paraphrasing and back translation (i.e., English $\to$ another language $\to$ English) attacks; and their ability to preserve quality and writing style of the unwatermarked texts by using linguistic metrics to capture quality and writing style of texts. Our results suggest that these watermarking techniques preserve semantics, deviate from the writing style of the unwatermarked texts, and are susceptible to adversarial attacks, especially for the back translation attack.

Method: RT obtains the final semantic representation by running multiple forward passes of the text embedding model, functioning as a test-time inference method.

Result: RT achieves significant improvements on semantic reasoning tasks in BRIGHT and PJBenchmark1, while maintaining consistent performance on general-purpose semantic understanding tasks like C-MTEB.

Conclusion: RT is effective because it further activates the semantic reasoning ability learned during pretraining by decoder-only text embedding models, serving as a successful test-time inference enhancement method.

Abstract: We propose RT (Refine Thought), a method that can enhance the semantic rea-soning ability of text embedding models. The method obtains the final semanticrepresentation by running multiple forward passes of the text embedding model.Experiments show that RT achieves significant improvements on semantic reason-ing tasks in BRIGHT and the person job matching benchmark PJBenchmark1, while maintaining consistent performance on general-purpose semantic under-standing tasks such as C-MTEB. Our results indicate that RT is effective becauseit further activates the semantic reasoning ability learned during pretraining bydecoder-only text embedding models(e.g., Qwen3-Embedding-8B). RT canbe seen as a test-time inference method.

Method: Two training-free approaches: 1) QE-informed Retranslation - selecting highest-quality translation from multiple LLM candidates, 2) QE-guided editing - instructing LLM to replace error substrings using QE explanations with conditional heuristic to minimize edits.

Result: The first approach achieved Delta COMET score of 0.0201 and won the WMT 2025 task, while the second approach scored -0.0108. The winning approach outperformed the QE-guided editing method.

Conclusion: QE-informed Retranslation (selecting best translation from multiple LLM candidates) is an effective training-free approach for error correction that avoids overcorrection issues of traditional APE systems.

Abstract: The paper presents two approaches submitted to the WMT 2025 Automated Translation Quality Evaluation Systems Task 3 - Quality Estimation (QE)-informed Segment-level Error Correction. While jointly training QE systems with Automatic Post-Editing (APE) has shown improved performance for both tasks, APE systems are still known to overcorrect the output of Machine Translation (MT), leading to a degradation in performance. We investigate a simple training-free approach - QE-informed Retranslation, and compare it with another within the same training-free paradigm. Our winning approach selects the highest-quality translation from multiple candidates generated by different LLMs. The second approach, more akin to APE, instructs an LLM to replace error substrings as specified in the provided QE explanation(s). A conditional heuristic was employed to minimise the number of edits, with the aim of maximising the Gain-to-Edit ratio. The two proposed approaches achieved a Delta COMET score of 0.0201 and -0.0108, respectively, leading the first approach to achieve the winning position on the subtask leaderboard.

Method: Created GM-Extract benchmark dataset, used two metrics (Document Metric for spatial retrieval and Variable Extraction Metric for semantic retrieval), evaluated 7-8B parameter models on multi-document tasks, and tested categorized mitigation methods (black-box and white-box).

Result: Performance varied significantly based on data representation in context window, with clear performance patterns across models correlated with perplexity scores. Mitigation methods showed highly nuanced efficacy - sometimes improving performance, sometimes having negative impact.

Conclusion: The study provides comprehensive understanding of mitigation strategies’ utility in practical contexts, highlighting that their effectiveness is highly dependent on specific scenarios and data representation approaches.

Abstract: The diminishing ability of large language models (LLMs) to effectively utilize long-range context-the “lost-in-the-middle” phenomenon-poses a significant challenge in retrieval-based LLM applications. To study the impact of this phenomenon in a real-world application setting, we introduce GM-Extract, a novel benchmark dataset meticulously designed to evaluate LLM performance on retrieval of control variables. To accurately diagnose failure modes, we propose a simple yet elegant evaluation system using two distinct metrics: one for spatial retrieval capability (Document Metric) and the other for semantic retrieval capability (Variable Extraction Metric). We conduct a systematic evaluation of 7-8B parameter models on two multi-document tasks (key-value extraction and question-answering), demonstrating a significant change in retrieval performance simply by altering how the data is represented in the context window. While a distinct U-shaped curve was not consistently observed, our analysis reveals a clear pattern of performance across models, which we further correlate with perplexity scores. Furthermore, we perform a literature survey of mitigation methods, which we categorize into two distinct approaches: black-box and white-box methods. We then apply these techniques to our benchmark, finding that their efficacy is highly nuanced. Our evaluation highlights scenarios where these strategies successfully improve performance, as well as surprising cases where they lead to a negative impact, providing a comprehensive understanding of their utility in a practical context.

Method: Decompose queries into attribute-value hints generated concurrently with retrieval, then transfer these interpretations to lightweight models to avoid LLM latency issues.

Result: Improves search performance by 10.9 points in MAP and ranking by 5.9 points in MRR over baselines.

Conclusion: The approach successfully represents and transfers superlative semantics between models, advancing linguistic interpretation in retrieval systems while maintaining practical deployment efficiency.

Abstract: Search queries with superlatives (e.g., best, most popular) require comparing candidates across multiple dimensions, demanding linguistic understanding and domain knowledge. We show that LLMs can uncover latent intent behind these expressions in e-commerce queries through a framework that extracts structured interpretations or hints. Our approach decomposes queries into attribute-value hints generated concurrently with retrieval, enabling efficient integration into the ranking pipeline. Our method improves search performanc eby 10.9 points in MAP and ranking by 5.9 points in MRR over baselines. Since direct LLM-based reranking faces prohibitive latency, we develop an efficient approach transferring superlative interpretations to lightweight models. Our findings provide insights into how superlative semantics can be represented and transferred between models, advancing linguistic interpretation in retrieval systems while addressing practical deployment constraints.

Method: Introduces Mixture-of-Retrieval Agentic RAG (MoRA-RAG) with dynamic query routing across hazard-specific databases, agentic chunking for contextual coherence, and verification loops for evidence assessment and query refinement.

Result: Achieves 94.5% accuracy on HazardRecQA dataset (derived from 90 global events across 7 hazard types), outperforming zero-shot LLMs by 30% and state-of-the-art RAG systems by 10%, while reducing hallucinations across LLM architectures.

Conclusion: Establishes a new paradigm for transforming post-disaster documentation into actionable, trustworthy intelligence for hazard resilience, enabling open-weight LLMs to match proprietary model performance.

Abstract: Post-disaster reconnaissance reports contain critical evidence for understanding multi-hazard interactions, yet their unstructured narratives make systematic knowledge transfer difficult. Large language models (LLMs) offer new potential for analyzing these reports, but often generate unreliable or hallucinated outputs when domain grounding is absent. This study introduces the Mixture-of-Retrieval Agentic RAG (MoRA-RAG), a knowledge-grounded LLM framework that transforms reconnaissance reports into a structured foundation for multi-hazard reasoning. The framework integrates a Mixture-of-Retrieval mechanism that dynamically routes queries across hazard-specific databases while using agentic chunking to preserve contextual coherence during retrieval. It also includes a verification loop that assesses evidence sufficiency, refines queries, and initiates targeted searches when information remains incomplete. We construct HazardRecQA by deriving question-answer pairs from GEER reconnaissance reports, which document 90 global events across seven major hazard types. MoRA-RAG achieves up to 94.5 percent accuracy, outperforming zero-shot LLMs by 30 percent and state-of-the-art RAG systems by 10 percent, while reducing hallucinations across diverse LLM architectures. MoRA-RAG also enables open-weight LLMs to achieve performance comparable to proprietary models. It establishes a new paradigm for transforming post-disaster documentation into actionable, trustworthy intelligence for hazard resilience.

Method: Proposes HiEAG framework with three key components: evidence retrieval, evidence reranking using Automatic Evidence Selection Prompting (AESP), and evidence rewriting using Automatic Evidence Generation Prompting (AEGP), leveraging MLLMs for comprehensive external consistency checking.

Result: Experimental results on benchmark datasets show that HiEAG surpasses previous state-of-the-art methods in accuracy across all samples, demonstrating superior performance in multimodal OOC misinformation detection.

Conclusion: The HiEAG framework effectively addresses the limitation of existing methods by incorporating external consistency checking through hierarchical evidence augmentation, achieving impressive performance with instruction tuning and enabling explanation for judgment.

Abstract: Recent advancements in multimodal out-of-context (OOC) misinformation detection have made remarkable progress in checking the consistencies between different modalities for supporting or refuting image-text pairs. However, existing OOC misinformation detection methods tend to emphasize the role of internal consistency, ignoring the significant of external consistency between image-text pairs and external evidence. In this paper, we propose HiEAG, a novel Hierarchical Evidence-Augmented Generation framework to refine external consistency checking through leveraging the extensive knowledge of multimodal large language models (MLLMs). Our approach decomposes external consistency checking into a comprehensive engine pipeline, which integrates reranking and rewriting, apart from retrieval. Evidence reranking module utilizes Automatic Evidence Selection Prompting (AESP) that acquires the relevant evidence item from the products of evidence retrieval. Subsequently, evidence rewriting module leverages Automatic Evidence Generation Prompting (AEGP) to improve task adaptation on MLLM-based OOC misinformation detectors. Furthermore, our approach enables explanation for judgment, and achieves impressive performance with instruction tuning. Experimental results on different benchmark datasets demonstrate that our proposed HiEAG surpasses previous state-of-the-art (SOTA) methods in the accuracy over all samples.

Method: Use vision-language models (VLMs) as judges to evaluate figures against dynamically generated domain-specific rubrics, treating plots as verifiable checkpoints for agents to correct errors and steer exploratory data analysis.

Result: VLM-augmented systems achieve pass@1 scores of 0.7-0.8 on a 10-task benchmark, significantly outperforming code-only (0.2-0.3) and code-and-text baselines (0.4-0.5), while providing auditable reasoning traces.

Conclusion: VLM-guided multi-agent systems effectively improve autonomous scientific discovery by enabling real-time error correction and adaptation, demonstrating superior performance over traditional approaches while maintaining interpretability.

Abstract: We show that multi-agent systems guided by vision-language models (VLMs) improve end-to-end autonomous scientific discovery. By treating plots as verifiable checkpoints, a VLM-as-a-judge evaluates figures against dynamically generated domain-specific rubrics, enabling agents to correct their own errors and steer exploratory data analysis in real-time. Case studies in cosmology and astrochemistry demonstrate recovery from faulty reasoning paths and adaptation to new datasets without human intervention. On a 10-task benchmark for data-driven discovery, VLM-augmented systems achieve pass at 1 scores of 0.7-0.8, compared to 0.2-0.3 for code-only and 0.4-0.5 for code-and-text baselines, while also providing auditable reasoning traces that improve interpretability. Code available here: https://github.com/CMBAgents/cmbagent

Method: Constructed balanced dataset by integrating GoEmotions, Sentiment140 samples, and GPT-4 generated texts. Built model with FastText embeddings, CNN, BiLSTM, attention mechanism, and mixed precision training.

Result: Significant improvements in accuracy, precision, recall, F1-score, and AUC compared to models trained on imbalanced data.

Conclusion: The approach effectively addresses class imbalance in multi-label sentiment classification and demonstrates superior performance.

Abstract: Multi-label sentiment classification plays a vital role in natural language processing by detecting multiple emotions within a single text. However, existing datasets like GoEmotions often suffer from severe class imbalance, which hampers model performance, especially for underrepresented emotions. To address this, we constructed a balanced multi-label sentiment dataset by integrating the original GoEmotions data, emotion-labeled samples from Sentiment140 using a RoBERTa-base-GoEmotions model, and manually annotated texts generated by GPT-4 mini. Our data balancing strategy ensured an even distribution across 28 emotion categories. Based on this dataset, we developed an enhanced multi-label classification model that combines pre-trained FastText embeddings, convolutional layers for local feature extraction, bidirectional LSTM for contextual learning, and an attention mechanism to highlight sentiment-relevant words. A sigmoid-activated output layer enables multi-label prediction, and mixed precision training improves computational efficiency. Experimental results demonstrate significant improvements in accuracy, precision, recall, F1-score, and AUC compared to models trained on imbalanced data, highlighting the effectiveness of our approach.

Method: Uses segment-level interference to elicit harmful reasoning traces, reuses self-generated outputs as supervised fine-tuning data with turn-based weighted loss design, and employs lightweight QLoRA fine-tuning.

Result: Achieves 38.52% higher attack success rate than IDEATOR with only 499 samples and under 3 hours on a single A100, while preserving general reasoning ability and original representation distribution.

Conclusion: Stealth Fine-Tuning is a low-cost, highly effective method to bypass alignment defenses in RVLMs, demonstrating significant vulnerabilities in current safety mechanisms.

Abstract: Reasoning-augmented Vision-Language Models (RVLMs) rely on safety alignment to prevent harmful behavior, yet their exposed chain-of-thought (CoT) traces introduce new attack surfaces. In this work, we find that the safety alignment of RVLMs can be easily break through a novel attack method termed \textbf{Stealth Fine-Tuning}. Our method elicits harmful reasoning traces through \textbf{segment-level interference} and reuses the self-generated outputs as supervised fine-tuning data. Through a \textbf{turn-based weighted} loss design, yielding a lightweight, distribution-consistent finetuning method. In our experiment, with only 499 samples and under 3 hours on a single A100 (QLoRA), Stealth Fine-Tuning outperforms IDEATOR by 38.52% ASR while preserving general reasoning ability, as the tuned model retains the original representation distribution. Experiments on AdvBench and several general benchmarks demonstrate that Stealth Fine-Tuning is a low-cost and highly effective way to bypass alignment defenses. \textcolor{red}{\textbf{Disclaimer: This paper contains content that may be disturbing or offensive.}}

Method: Constructs realistic multi-label code sets using real-world co-occurrence patterns, ICD descriptions, synonyms, taxonomy, and similar clinical notes to generate 90,000 synthetic discharge summaries covering 7,902 ICD codes.

Result: Fine-tuning PLM-ICD and GKI-ICD models on extended datasets shows modest macro-F1 improvements while maintaining strong micro-F1, outperforming prior state-of-the-art methods.

Conclusion: Carefully crafted synthetic data can enhance equity in long-tail ICD code prediction, though gains may be marginal relative to computational costs.

Abstract: Automatic ICD coding from clinical text is a critical task in medical NLP but remains hindered by the extreme long-tail distribution of diagnostic codes. Thousands of rare and zero-shot ICD codes are severely underrepresented in datasets like MIMIC-III, leading to low macro-F1 scores. In this work, we propose a data-centric framework that generates high-quality synthetic discharge summaries to mitigate this imbalance. Our method constructs realistic multi-label code sets anchored on rare codes by leveraging real-world co-occurrence patterns, ICD descriptions, synonyms, taxonomy, and similar clinical notes. Using these structured prompts, we generate 90,000 synthetic notes covering 7,902 ICD codes, significantly expanding the training distribution. We fine-tune two state-of-the-art transformer-based models, PLM-ICD and GKI-ICD, on both the original and extended datasets. Experiments show that our approach modestly improves macro-F1 while maintaining strong micro-F1, outperforming prior SOTA. While the gain may seem marginal relative to the computational cost, our results demonstrate that carefully crafted synthetic data can enhance equity in long-tail ICD code prediction.

Method: Dynamic hypergraph framework with sample-specific hierarchical clustering using a novel acceleration-fallback cutoff to adaptively determine hyperedge granularity, modeling aspect-opinion structures more efficiently.

Result: Consistent improvements over strong graph baselines on three benchmarks (Lap14, Rest14, MAMS), with substantial gains when paired with RoBERTa backbones.

Conclusion: Dynamic hypergraph construction is an efficient and powerful alternative for ABSA with potential extensions to other short-text NLP tasks.

Abstract: Aspect-Based Sentiment Analysis (ABSA) predicts sentiment polarity for specific aspect terms, a task made difficult by conflicting sentiments across aspects and the sparse context of short texts. Prior graph-based approaches model only pairwise dependencies, forcing them to construct multiple graphs for different relational views. These introduce redundancy, parameter overhead, and error propagation during fusion, limiting robustness in short-text, low-resource settings. We present HyperABSA, a dynamic hypergraph framework that induces aspect-opinion structures through sample-specific hierarchical clustering. To construct these hyperedges, we introduce a novel acceleration-fallback cutoff for hierarchical clustering, which adaptively determines the level of granularity. Experiments on three benchmarks (Lap14, Rest14, MAMS) show consistent improvements over strong graph baselines, with substantial gains when paired with RoBERTa backbones. These results position dynamic hypergraph construction as an efficient, powerful alternative for ABSA, with potential extensions to other short-text NLP tasks.

Method: Proposes a method that: (i) converts question sentences into relational graphs using Transformer-based relation extraction, and (ii) verifies them against factually correct knowledge graphs under closed-world assumption.

Result: The method correctly answers up to around 70% of questions while providing traceability of the procedure, with question category having significant influence on accuracy.

Conclusion: Transformer-based relation extraction combined with knowledge graph matching provides an effective approach for MCQ answering with traceable reasoning, though performance varies by question type.

Abstract: In this research, we combine Transformer-based relation extraction with matching of knowledge graphs (KGs) and apply them to answering multiple-choice questions (MCQs) while maintaining the traceability of the output process. KGs are structured representations of factual knowledge consisting of entities and relations. Due to the high construction cost, they had been regarded as static databases with validated links. However, the recent development of Transformer-based relation extraction (RE) methods has enabled us to generate KGs dynamically by giving them natural language texts, and thereby opened the possibility for representing the meaning of the input sentences with the created KGs. Using this effect, we propose a method that answers MCQs in the “fill-in-the-blank” format, taking care of the point that RE methods generate KGs that represent false information if provided with factually incorrect texts. We measure the truthfulness of each question sentence by (i) converting the sentence into a relational graph using an RE method and (ii) verifying it against factually correct KGs under the closed-world assumption. The experimental results demonstrate that our method correctly answers up to around 70% of the questions, while providing traceability of the procedure. We also highlight that the question category has a vast influence on the accuracy.

Method: Trains a binary classifier P(IK) to identify questions that strong models can answer, uses self-generated labels for alignment, and refines weak labels with graph smoothing.

Result: Extensive experiments on three benchmarks show consistent outperformance over competitive baselines, with P(IK) demonstrating generalization across tasks and difficulties.

Conclusion: Selective weak-to-strong generalization can help superalignment by avoiding harmful weak supervision while leveraging model capabilities effectively.

Abstract: Future superhuman models will surpass the ability of humans and humans will only be able to \textit{weakly} supervise superhuman models. To alleviate the issue of lacking high-quality data for model alignment, some works on weak-to-strong generalization (W2SG) finetune a strong pretrained model with a weak supervisor so that it can generalize beyond weak supervision. However, the invariable use of weak supervision in existing methods exposes issues in robustness, with a proportion of weak labels proving harmful to models. In this paper, we propose a selective W2SG framework to avoid using weak supervision when unnecessary. We train a binary classifier P(IK) to identify questions that a strong model can answer and use its self-generated labels for alignment. We further refine weak labels with a graph smoothing method. Extensive experiments on three benchmarks show that our method consistently outperforms competitive baselines. Further analyses show that P(IK) can generalize across tasks and difficulties, which indicates selective W2SG can help superalignment.

Method: Proposed symbolic localization framework using symbolic linguistic and semantic knowledge to analyze five models on HaluEval and TruthfulQA, focusing on attention variance for symbolic triggers.

Result: Attention variance for symbolic elements explodes in early layers (2-4), with negation causing catastrophic variance. Hallucination rates remain high (78.3%-83.7%) despite model size, with steep attention drops for symbolic triggers in deeper layers.

Conclusion: Hallucination is fundamentally a symbolic linguistic processing failure, not general generation problem. Symbolic semantic knowledge is key to understanding and localizing hallucination mechanisms in LLMs.

Abstract: LLMs still struggle with hallucination, especially when confronted with symbolic triggers like modifiers, negation, numbers, exceptions, and named entities. Yet, we lack a clear understanding of where these symbolic hallucinations originate, making it crucial to systematically handle such triggers and localize the emergence of hallucination inside the model. While prior work explored localization using statistical techniques like LSC and activation variance analysis, these methods treat all tokens equally and overlook the role symbolic linguistic knowledge plays in triggering hallucinations. So far, no approach has investigated how symbolic elements specifically drive hallucination failures across model layers, nor has symbolic linguistic knowledge been used as the foundation for a localization framework. We propose the first symbolic localization framework that leverages symbolic linguistic and semantic knowledge to meaningfully trace the development of hallucinations across all model layers. By focusing on how models process symbolic triggers, we analyze five models using HaluEval and TruthfulQA. Our symbolic knowledge approach reveals that attention variance for these linguistic elements explodes to critical instability in early layers (2-4), with negation triggering catastrophic variance levels, demonstrating that symbolic semantic processing breaks down from the very beginning. Through the lens of symbolic linguistic knowledge, despite larger model sizes, hallucination rates remain consistently high (78.3%-83.7% across Gemma variants), with steep attention drops for symbolic semantic triggers throughout deeper layers. Our findings demonstrate that hallucination is fundamentally a symbolic linguistic processing failure, not a general generation problem, revealing that symbolic semantic knowledge provides the key to understanding and localizing hallucination mechanisms in LLMs.

Method: DeepEL incorporates LLMs into every stage of entity linking and introduces a self-validation mechanism that utilizes global contextual information to enable LLMs to correct their own predictions and better understand cohesive relationships among entities within sentences.

Result: DeepEL achieves substantial improvements over state-of-the-art methods across ten benchmark datasets, with an average 2.6% F1 score improvement and a remarkable 4% gain on out-of-domain datasets.

Conclusion: Deep integration of LLMs throughout the entity linking pipeline, combined with self-validation mechanisms, significantly advances the state-of-the-art in entity linking performance.

Abstract: Entity Linking (EL), the task of mapping textual entity mentions to their corresponding entries in knowledge bases, constitutes a fundamental component of natural language understanding. Recent advancements in Large Language Models (LLMs) have demonstrated remarkable potential for enhancing EL performance. Prior research has leveraged LLMs to improve entity disambiguation and input representation, yielding significant gains in accuracy and robustness. However, these approaches typically apply LLMs to isolated stages of the EL task, failing to fully integrate their capabilities throughout the entire process. In this work, we introduce DeepEL, a comprehensive framework that incorporates LLMs into every stage of the entity linking task. Furthermore, we identify that disambiguating entities in isolation is insufficient for optimal performance. To address this limitation, we propose a novel self-validation mechanism that utilizes global contextual information, enabling LLMs to rectify their own predictions and better recognize cohesive relationships among entities within the same sentence. Extensive empirical evaluation across ten benchmark datasets demonstrates that DeepEL substantially outperforms existing state-of-the-art methods, achieving an average improvement of 2.6% in overall F1 score and a remarkable 4% gain on out-of-domain datasets. These results underscore the efficacy of deep LLM integration in advancing the state-of-the-art in entity linking.

Method: ArbESC+ framework collects correction proposals from multiple models, represents them as numerical features, uses classifier to select corrections, with support techniques for filtering and reliability estimation.

Result: Achieved F0.5 scores: 82.63% on QALB-14, 84.64% on QALB-15 L1, 65.55% on QALB-15 L2, outperforming single models.

Conclusion: First Arabic attempt to integrate linguistic error correction, providing practical advancement for Arabic text processing tools and benefiting users/researchers.

Abstract: Grammatical Error Correction (GEC) is an important aspect of natural language processing. Arabic has a complicated morphological and syntactic structure, posing a greater challenge than other languages. Even though modern neural models have improved greatly in recent years, the majority of previous attempts used individual models without taking into account the potential benefits of combining different systems. In this paper, we present one of the first multi-system approaches for correcting grammatical errors in Arabic, the Arab Enhanced Edit Selection System Complication (ArbESC+). Several models are used to collect correction proposals, which are represented as numerical features in the framework. A classifier determines and implements the appropriate corrections based on these features. In order to improve output quality, the framework uses support techniques to filter overlapping corrections and estimate decision reliability. A combination of AraT5, ByT5, mT5, AraBART, AraBART+Morph+GEC, and Text editing systems gave better results than a single model alone, with F0.5 at 82.63% on QALB-14 test data, 84.64% on QALB-15 L1 data, and 65.55% on QALB-15 L2 data. As one of the most significant contributions of this work, it’s the first Arab attempt to integrate linguistic error correction. Improving existing models provides a practical step towards developing advanced tools that will benefit users and researchers of Arabic text processing.

Method: Constructed a large music corpus combining open source and in-house data with domain-first pipeline (classifier filtering, multi-stage cleaning, de-duplication, privacy masking). Used reference-model-based token-level soft scoring for quality control with unified loss-ratio criterion for data selection and dynamic down-weighting during optimization.

Result: Developed MusicSimpleQA benchmark for factuality assessment with short, single-answer prompts and automated agreement scoring. Created scalable data-training framework and reusable evaluation tool for music domain LLMs.

Conclusion: The work advances both corpus construction and training objectives, providing a scalable framework for building effective domain-specific LLMs in music with improved data quality control and evaluation methodology.

Abstract: Large language models perform strongly on general tasks but remain constrained in specialized settings such as music, particularly in the music-entertainment domain, where corpus scale, purity, and the match between data and training objectives are critical. We address this by constructing a large, music-related natural language corpus (40B tokens) that combines open source and in-house data, and by implementing a domain-first data pipeline: a lightweight classifier filters and weights in-domain text, followed by multi-stage cleaning, de-duplication, and privacy-preserving masking. We further integrate multi-source music text with associated metadata to form a broader, better-structured foundation of domain knowledge. On the training side, we introduce reference-model (RM)-based token-level soft scoring for quality control: a unified loss-ratio criterion is used both for data selection and for dynamic down-weighting during optimization, reducing noise gradients and amplifying task-aligned signals, thereby enabling more effective music-domain continued pretraining and alignment. To assess factuality, we design the MusicSimpleQA benchmark, which adopts short, single-answer prompts with automated agreement scoring. Beyond the benchmark design, we conduct systematic comparisons along the axes of data composition. Overall, this work advances both the right corpus and the right objective, offering a scalable data-training framework and a reusable evaluation tool for building domain LLMs in the music field.

Method: Proposes Retrieve-Augmented Director-Actor Interaction Learning with three mechanisms: multimodal Reference Footage library with LLM comprehension, Emotion-Similarity-based Retrieval-Augmentation, and Progressive Graph-based speech generation.

Result: Achieves comprehensive improvements in emotional expressiveness, validated by both subjective and objective evaluations on the V2C Animation benchmark dataset.

Conclusion: The proposed approach successfully replicates authentic dubbing workflow and outperforms existing methods in emotional expressiveness for movie dubbing.

Abstract: The automatic movie dubbing model generates vivid speech from given scripts, replicating a speaker’s timbre from a brief timbre prompt while ensuring lip-sync with the silent video. Existing approaches simulate a simplified workflow where actors dub directly without preparation, overlooking the critical director-actor interaction. In contrast, authentic workflows involve a dynamic collaboration: directors actively engage with actors, guiding them to internalize the context cues, specifically emotion, before performance. To address this issue, we propose a new Retrieve-Augmented Director-Actor Interaction Learning scheme to achieve authentic movie dubbing, termed Authentic-Dubber, which contains three novel mechanisms: (1) We construct a multimodal Reference Footage library to simulate the learning footage provided by directors. Note that we integrate Large Language Models (LLMs) to achieve deep comprehension of emotional representations across multimodal signals. (2) To emulate how actors efficiently and comprehensively internalize director-provided footage during dubbing, we propose an Emotion-Similarity-based Retrieval-Augmentation strategy. This strategy retrieves the most relevant multimodal information that aligns with the target silent video. (3) We develop a Progressive Graph-based speech generation approach that incrementally incorporates the retrieved multimodal emotional knowledge, thereby simulating the actor’s final dubbing process. The above mechanisms enable the Authentic-Dubber to faithfully replicate the authentic dubbing workflow, achieving comprehensive improvements in emotional expressiveness. Both subjective and objective evaluations on the V2C Animation benchmark dataset validate the effectiveness. The code and demos are available at https://github.com/AI-S2-Lab/Authentic-Dubber.

Method: Created a comprehensive benchmark using spontaneous and non-spontaneous speech conversations from various open African accented English speech datasets, evaluating open, closed, unimodal ASR and multimodal LLM-based speech recognition systems across 7 domains.

Result: Open-source ASR models excel in spontaneous speech but degrade on noisy dialogue; multimodal LLMs are accent-robust but struggle with domain-specific entities; proprietary models deliver high accuracy on clean speech but vary by country/domain; African English fine-tuned models achieve competitive accuracy with lower latency.

Conclusion: The benchmark empowers practitioners and researchers to select voice technologies suited to African use-cases, fostering inclusive voice applications for underserved communities, though hallucinations remain a significant problem for most state-of-the-art models.

Abstract: Recent advances in speech-enabled AI, including Google’s NotebookLM and OpenAI’s speech-to-speech API, are driving widespread interest in voice interfaces globally. Despite this momentum, there exists no publicly available application-specific model evaluation that caters to Africa’s linguistic diversity. We present AfriSpeech-MultiBench, the first domain-specific evaluation suite for over 100 African English accents across 10+ countries and seven application domains: Finance, Legal, Medical, General dialogue, Call Center, Named Entities and Hallucination Robustness. We benchmark a diverse range of open, closed, unimodal ASR and multimodal LLM-based speech recognition systems using both spontaneous and non-spontaneous speech conversation drawn from various open African accented English speech datasets. Our empirical analysis reveals systematic variation: open-source ASR models excels in spontaneous speech contexts but degrades on noisy, non-native dialogue; multimodal LLMs are more accent-robust yet struggle with domain-specific named entities; proprietary models deliver high accuracy on clean speech but vary significantly by country and domain. Models fine-tuned on African English achieve competitive accuracy with lower latency, a practical advantage for deployment, hallucinations still remain a big problem for most SOTA models. By releasing this comprehensive benchmark, we empower practitioners and researchers to select voice technologies suited to African use-cases, fostering inclusive voice applications for underserved communities.

Method: Uses entropy-guided training framework that guides model toward efficient reasoning as entropy decreases and reinforces exploration as entropy rises under compact reasoning mode.

Result: Compresses reasoning length to 20% of original while maintaining or surpassing baseline accuracy on six mathematical benchmarks.

Conclusion: The entropy-guided approach effectively resolves entropy conflict in compression training, enabling significant reasoning compression without accuracy loss.

Abstract: Large reasoning models have demonstrated remarkable performance on complex reasoning tasks, yet the excessive length of their chain-of-thought outputs remains a major practical bottleneck due to high computation cost and poor deployability. Existing compression methods have achieved partial success but overlook a crucial phenomenon in the training process – the entropy conflict. During compression training, entropy decreases, leading to shorter reasoning but limited exploration, while accuracy-oriented objectives increase entropy, lengthening reasoning chains. This can cause the model to get stuck in a local dilemma. Our analysis further reveals the origin of the entropy conflict: many high-entropy tokens are logical connectors that receive larger gradients and are encouraged under the performance objective, while the compression objective simultaneously penalizes these potentially redundant connectors. This opposing pressure creates a direct source of entropy conflict. To address these issues, we adopt an entropy-guided training framework. As entropy descends, the model is guided toward efficient reasoning by encouraging concise thought steps; as entropy rises, exploration is reinforced under the compact reasoning mode to improve robustness. Experiments on six mathematical benchmarks show that our method compresses reasoning length to 20% of the original while maintaining or even surpassing baseline accuracy. Code and models will be released publicly.

Method: Proposes predicting verbalized probability distributions over all possible answers, which forces LLMs to consider all candidates in the answer space and carefully assign confidence scores that form a valid distribution.

Result: The method shows advantages across different models and tasks, works regardless of whether the answer space is known, maintains benefits after reinforcement learning, and produces reasoning patterns aligned with human expectations.

Conclusion: Predicting probability distributions is an effective approach for enhancing LLM reasoning in confidence estimation, providing more reliable and transparent confidence assessments.

Abstract: Knowing the reliability of a model’s response is essential in application. With the strong generation capabilities of LLMs, research has focused on generating verbalized confidence. This is further enhanced by combining chain-of-thought reasoning, which provides logical and transparent estimation. However, how reasoning strategies affect the estimated confidence is still under-explored. In this work, we demonstrate that predicting a verbalized probability distribution can effectively encourage in-depth reasoning for confidence estimation. Intuitively, it requires an LLM to consider all candidates within the answer space instead of basing on a single guess, and to carefully assign confidence scores to meet the requirements of a distribution. This method shows an advantage across different models and various tasks, regardless of whether the answer space is known. Its advantage is maintained even after reinforcement learning, and further analysis shows its reasoning patterns are aligned with human expectations.

Method: Created a fully human-annotated benchmark with 150 expert-designed multiple choice questions spanning grammar, morphology, spelling, reading comprehension, and syntax, then evaluated 35 Arabic and bilingual LLMs.

Result: Current models show strong surface level proficiency but struggle with deeper grammatical and syntactic reasoning, succeeding through memorization or pattern recognition rather than authentic comprehension.

Conclusion: AraLingBench provides a diagnostic framework for developing Arabic LLMs by isolating and measuring fundamental linguistic skills, highlighting the need for models that achieve true linguistic mastery beyond surface-level performance.

Abstract: We present AraLingBench: a fully human annotated benchmark for evaluating the Arabic linguistic competence of large language models (LLMs). The benchmark spans five core categories: grammar, morphology, spelling, reading comprehension, and syntax, through 150 expert-designed multiple choice questions that directly assess structural language understanding. Evaluating 35 Arabic and bilingual LLMs reveals that current models demonstrate strong surface level proficiency but struggle with deeper grammatical and syntactic reasoning. AraLingBench highlights a persistent gap between high scores on knowledge-based benchmarks and true linguistic mastery, showing that many models succeed through memorization or pattern recognition rather than authentic comprehension. By isolating and measuring fundamental linguistic skills, AraLingBench provides a diagnostic framework for developing Arabic LLMs. The full evaluation code is publicly available on GitHub.

Method: Introduce ConInstruct benchmark to assess LLMs’ conflict detection and resolution abilities, evaluating performance on conflict detection and analyzing conflict resolution behavior.

Result: Most proprietary LLMs show strong conflict detection (DeepSeek-R1: 91.5% F1, Claude-4.5-Sonnet: 87.3% F1), but LLMs rarely explicitly notify users about conflicts or request clarification despite detection capabilities.

Conclusion: Current LLMs have a critical shortcoming in not communicating detected conflicts to users, highlighting an important area for future improvement in instruction-following LLM design.

Abstract: Instruction-following is a critical capability of Large Language Models (LLMs). While existing works primarily focus on assessing how well LLMs adhere to user instructions, they often overlook scenarios where instructions contain conflicting constraints-a common occurrence in complex prompts. The behavior of LLMs under such conditions remains under-explored. To bridge this gap, we introduce ConInstruct, a benchmark specifically designed to assess LLMs’ ability to detect and resolve conflicts within user instructions. Using this dataset, we evaluate LLMs’ conflict detection performance and analyze their conflict resolution behavior. Our experiments reveal two key findings: (1) Most proprietary LLMs exhibit strong conflict detection capabilities, whereas among open-source models, only DeepSeek-R1 demonstrates similarly strong performance. DeepSeek-R1 and Claude-4.5-Sonnet achieve the highest average F1-scores at 91.5% and 87.3%, respectively, ranking first and second overall. (2) Despite their strong conflict detection abilities, LLMs rarely explicitly notify users about the conflicts or request clarification when faced with conflicting constraints. These results underscore a critical shortcoming in current LLMs and highlight an important area for future improvement when designing instruction-following LLMs.

Method: Targeted vocabulary extension by adding chemically salient tokens to a pretrained LLM’s vocabulary, followed by continued pretraining on chemistry-domain text to integrate the new knowledge.

Result: The methodology leads to superior performance on a range of downstream chemical tasks, demonstrating the effectiveness of the approach.

Conclusion: Unifying natural language and molecular representations through vocabulary extension and domain-specific pretraining effectively resolves the tokenization bottleneck in chemistry applications of LLMs.

Abstract: The application of large language models (LLMs) to chemistry is frequently hampered by a “tokenization bottleneck”, where tokenizers tuned on general-domain text tend to fragment chemical representations such as SMILES into semantically uninformative sub-tokens. This paper introduces a principled methodology to resolve this bottleneck by unifying the representation of natural language and molecular structures within a single model. Our approach involves targeted vocabulary extension-augmenting a pretrained LLM’s vocabulary with chemically salient tokens, followed by continued pretraining on chemistry-domain text to integrate this new knowledge. We provide an empirical demonstration of the effectiveness of this strategy, showing that our methodology leads to superior performance on a range of downstream chemical tasks.

Method: Created 800 original problems across 7 scientific fields by domain experts, featuring contamination resistance, cross-disciplinary focus, complex open-ended answers, and rigorous quality control with expert peer review.

Result: Preliminary results show ATLAS effectively differentiates advanced scientific reasoning capabilities of leading models.

Conclusion: ATLAS will serve as a long-term, open platform to reliably measure progress toward Artificial General Intelligence.

Abstract: The rapid advancement of Large Language Models (LLMs) has led to performance saturation on many established benchmarks, questioning their ability to distinguish frontier models. Concurrently, existing high-difficulty benchmarks often suffer from narrow disciplinary focus, oversimplified answer formats, and vulnerability to data contamination, creating a fidelity gap with real-world scientific inquiry. To address these challenges, we introduce ATLAS (AGI-Oriented Testbed for Logical Application in Science), a large-scale, high-difficulty, and cross-disciplinary evaluation suite composed of approximately 800 original problems. Developed by domain experts (PhD-level and above), ATLAS spans seven core scientific fields: mathematics, physics, chemistry, biology, computer science, earth science, and materials science. Its key features include: (1) High Originality and Contamination Resistance, with all questions newly created or substantially adapted to prevent test data leakage; (2) Cross-Disciplinary Focus, designed to assess models’ ability to integrate knowledge and reason across scientific domains; (3) High-Fidelity Answers, prioritizing complex, open-ended answers involving multi-step reasoning and LaTeX-formatted expressions over simple multiple-choice questions; and (4) Rigorous Quality Control, employing a multi-stage process of expert peer review and adversarial testing to ensure question difficulty, scientific value, and correctness. We also propose a robust evaluation paradigm using a panel of LLM judges for automated, nuanced assessment of complex answers. Preliminary results on leading models demonstrate ATLAS’s effectiveness in differentiating their advanced scientific reasoning capabilities. We plan to develop ATLAS into a long-term, open, community-driven platform to provide a reliable “ruler” for progress toward Artificial General Intelligence.

Method: Developed normalized contextual calibration (NCC), a method that normalizes and calibrates predictions at the full-label level rather than individual tokens.

Result: NCC achieves statistically significant improvements over prior approaches with gains up to 10% F1, extends to multiple-choice QA, and provides more reliable confidence estimates with fewer examples.

Conclusion: Mitigating full-label biases is crucial for improving LLM performance and robustness, especially in real-world applications with multi-token class labels.

Abstract: Large language models (LLMs) are powerful zero- and few-shot learners. However, when predicting over a set of candidate options, LLMs suffer from label biases, and existing calibration methods overlook biases arising from multi-token class labels. We tackle an issue we call label length bias, where labels of different lengths are treated inconsistently, even after standard length normalization. To mitigate it, we propose normalized contextual calibration (NCC), an effective method that normalizes and calibrates predictions at the full-label level. NCC achieves statistically significant improvements over prior approaches across multiple datasets and models, with gains of up to 10% F1. Moreover, NCC extends bias mitigation to broader tasks such as multiple-choice question answering. Our analysis shows that, when combined with in-context learning, NCC is less sensitive to few-shot example selection, requires fewer examples for competitive performance, and produces more reliable confidence estimates. These findings highlight the importance of mitigating full-label biases to improve the performance and robustness of LLM-based methods, particularly in real-world applications where class labels naturally consist of multiple tokens.

Method: Three-stage shield framework: 1) safety-aware attention realignment, 2) layer-wise safety judgment, 3) defense-driven dual routing to separate safe/unsafe queries.

Result: TSSF effectively strengthens safety against 8 jailbreak strategies while preventing over-refusal, and provides robust defense against fine-tuning attacks while maintaining utility.

Conclusion: The proposed framework successfully addresses educational LLM safety vulnerabilities through systematic evaluation and multi-stage defense mechanisms.

Abstract: Large Language Models (LLMs) are increasingly integrated into educational applications. However, they remain vulnerable to jailbreak and fine-tuning attacks, which can compromise safety alignment and lead to harmful outputs. Existing studies mainly focus on general safety evaluations, with limited attention to the unique safety requirements of educational scenarios. To address this gap, we construct EduHarm, a benchmark containing safe-unsafe instruction pairs across five representative educational scenarios, enabling systematic safety evaluation of educational LLMs. Furthermore, we propose a three-stage shield framework (TSSF) for educational LLMs that simultaneously mitigates both jailbreak and fine-tuning attacks. First, safety-aware attention realignment redirects attention toward critical unsafe tokens, thereby restoring the harmfulness feature that discriminates between unsafe and safe inputs. Second, layer-wise safety judgment identifies harmfulness features by aggregating safety cues across multiple layers to detect unsafe instructions. Finally, defense-driven dual routing separates safe and unsafe queries, ensuring normal processing for benign inputs and guarded responses for harmful ones. Extensive experiments across eight jailbreak attack strategies demonstrate that TSSF effectively strengthens safety while preventing over-refusal of benign queries. Evaluations on three fine-tuning attack datasets further show that it consistently achieves robust defense against harmful queries while maintaining preserving utility gains from benign fine-tuning.

Method: Developed a cloud-based benchmarking infrastructure with expert-curated tasks reviewed by clinicians from 500+ institutions, using LLM-as-a-judge scoring calibrated to human ratings, and evaluating 15 frontier models across LLM, multimodal, and agent tracks.

Result: Base LLMs scored 54.1/100 (best: Claude Sonnet 4.5 at 62.5/100) with poor safety/ethics (18.4/100). Multimodal models averaged 47.5/100 (best: GPT-5 at 54.9/100) with weak cross-modal reasoning. Agents significantly outperformed with 79.8/100 average, with Claude Sonnet 4.5-based agents achieving 85.3/100 overall and 88.9/100 on safety.

Conclusion: Agentic orchestration substantially enhances clinical readiness without sacrificing capability, while base models still have gaps in multimodal reasoning and safety. The platform provides practical reference for medical AI auditing aligned with clinical guidelines.

Abstract: Recent advances in medical large language models (LLMs), multimodal models, and agents demand evaluation frameworks that reflect real clinical workflows and safety constraints. We present MedBench v4, a nationwide, cloud-based benchmarking infrastructure comprising over 700,000 expert-curated tasks spanning 24 primary and 91 secondary specialties, with dedicated tracks for LLMs, multimodal models, and agents. Items undergo multi-stage refinement and multi-round review by clinicians from more than 500 institutions, and open-ended responses are scored by an LLM-as-a-judge calibrated to human ratings. We evaluate 15 frontier models. Base LLMs reach a mean overall score of 54.1/100 (best: Claude Sonnet 4.5, 62.5/100), but safety and ethics remain low (18.4/100). Multimodal models perform worse overall (mean 47.5/100; best: GPT-5, 54.9/100), with solid perception yet weaker cross-modal reasoning. Agents built on the same backbones substantially improve end-to-end performance (mean 79.8/100), with Claude Sonnet 4.5-based agents achieving up to 85.3/100 overall and 88.9/100 on safety tasks. MedBench v4 thus reveals persisting gaps in multimodal reasoning and safety for base models, while showing that governance-aware agentic orchestration can markedly enhance benchmarked clinical readiness without sacrificing capability. By aligning tasks with Chinese clinical guidelines and regulatory priorities, the platform offers a practical reference for hospitals, developers, and policymakers auditing medical AI.

Method: Three-component system: (1) RAG assistant for personalized dialogue, (2) synthetic client-therapist dialogue generator for research and data augmentation, (3) Well-being AI crew using CrewAI for weekly self-care plans and meditation audio.

Result: System provides accessible mental health support, generates synthetic therapeutic data, and enables dynamic self-care planning. Evaluated through LLM-based judgments and human-user studies in well-being scenarios.

Conclusion: Demonstrates how conversational AI can complement mental health care and enable interdisciplinary collaboration between NLP and mental health professionals for responsible innovation.

Abstract: We present Tell Me, a mental well-being system that leverages advances in large language models to provide accessible, context-aware support for users and researchers. The system integrates three components: (i) a retrieval-augmented generation (RAG) assistant for personalized, knowledge-grounded dialogue; (ii) a synthetic client-therapist dialogue generator conditioned on client profiles to facilitate research on therapeutic language and data augmentation; and (iii) a Well-being AI crew, implemented with CrewAI, that produces weekly self-care plans and guided meditation audio. The system is designed as a reflective space for emotional processing rather than a substitute for professional therapy. It illustrates how conversational assistants can lower barriers to support, complement existing care, and broaden access to mental health resources. To address the shortage of confidential therapeutic data, we introduce synthetic client-therapist dialogue generation conditioned on client profiles. Finally, the planner demonstrates an innovative agentic workflow for dynamically adaptive, personalized self-care, bridging the limitations of static well-being tools. We describe the architecture, demonstrate its functionalities, and report evaluation of the RAG assistant in curated well-being scenarios using both automatic LLM-based judgments and a human-user study. This work highlights opportunities for interdisciplinary collaboration between NLP researchers and mental health professionals to advance responsible innovation in human-AI interaction for well-being.

Method: Systematically extend MDP framework to define LLM Agent components; introduce Agent-R1, a modular and user-friendly RL training framework for LLM Agents.

Result: Experiments on Multihop QA benchmark tasks provide initial validation of the proposed methods and framework effectiveness.

Conclusion: The paper advances RL methodologies for LLM Agents through systematic framework definition and practical training tools, showing promise for complex problem-solving.

Abstract: Large Language Models (LLMs) are increasingly being explored for building Agents capable of active environmental interaction (e.g., via tool use) to solve complex problems. Reinforcement Learning (RL) is considered a key technology with significant potential for training such Agents; however, the effective application of RL to LLM Agents is still in its nascent stages and faces considerable challenges. Currently, this emerging field lacks in-depth exploration into RL approaches specifically tailored for the LLM Agent context, alongside a scarcity of flexible and easily extensible training frameworks designed for this purpose. To help advance this area, this paper first revisits and clarifies Reinforcement Learning methodologies for LLM Agents by systematically extending the Markov Decision Process (MDP) framework to comprehensively define the key components of an LLM Agent. Secondly, we introduce Agent-R1, a modular, flexible, and user-friendly training framework for RL-based LLM Agents, designed for straightforward adaptation across diverse task scenarios and interactive environments. We conducted experiments on Multihop QA benchmark tasks, providing initial validation for the effectiveness of our proposed methods and framework.

Method: Created a synthetic dataset derived from SIGIR'2025 LiveRAG Challenge, augmented with ground-truth answers, supporting claims, and difficulty/discriminability scores using Item Response Theory.

Result: The benchmark shows diverse question types, wide range of difficulty levels, and effectively differentiates between system capabilities.

Conclusion: LiveRAG benchmark will help advance RAG research, enable systematic evaluation, and support development of more robust Q&A systems.

Abstract: With Retrieval Augmented Generation (RAG) becoming more and more prominent in generative AI solutions, there is an emerging need for systematically evaluating their effectiveness. We introduce the LiveRAG benchmark, a publicly available dataset of 895 synthetic questions and answers designed to support systematic evaluation of RAG-based Q&A systems. This synthetic benchmark is derived from the one used during the SIGIR'2025 LiveRAG Challenge, where competitors were evaluated under strict time constraints. It is augmented with information that was not made available to competitors during the Challenge, such as the ground-truth answers, together with their associated supporting claims which were used for evaluating competitors’ answers. In addition, each question is associated with estimated difficulty and discriminability scores, derived from applying an Item Response Theory model to competitors’ responses. Our analysis highlights the benchmark’s questions diversity, the wide range of their difficulty levels, and their usefulness in differentiating between system capabilities. The LiveRAG benchmark will hopefully help the community advance RAG research, conduct systematic evaluation, and develop more robust Q&A systems.

Method: The authors explore approaches to align two sets of claims from the same source document and compute similarity scores. They investigate techniques for optimal alignment and evaluation between claim sets, using a dataset of claims extracted from Czech and Slovak news article comments.

Result: The experiments reveal limitations of current evaluation approaches when applied to document-level claim extraction, particularly in handling informal language, local context, and language subtleties.

Conclusion: There is a need for more advanced evaluation methods that can properly capture semantic similarity and assess essential claim properties like atomicity, checkworthiness, and decontextualization.

Abstract: Document-level claim extraction remains an open challenge in the field of fact-checking, and subsequently, methods for evaluating extracted claims have received limited attention. In this work, we explore approaches to aligning two sets of claims pertaining to the same source document and computing their similarity through an alignment score. We investigate techniques to identify the best possible alignment and evaluation method between claim sets, with the aim of providing a reliable evaluation framework. Our approach enables comparison between model-extracted and human-annotated claim sets, serving as a metric for assessing the extraction performance of models and also as a possible measure of inter-annotator agreement. We conduct experiments on newly collected dataset-claims extracted from comments under Czech and Slovak news articles-domains that pose additional challenges due to the informal language, strong local context, and subtleties of these closely related languages. The results draw attention to the limitations of current evaluation approaches when applied to document-level claim extraction and highlight the need for more advanced methods-ones able to correctly capture semantic similarity and evaluate essential claim properties such as atomicity, checkworthiness, and decontextualization.

Method: Developed an automatic process for collecting naturally occurring summaries via front-page teasers where editors summarize full articles, scalable across varying linguistic resource levels.

Result: Successfully applied the process to Hebrew newspapers, producing HEBTEASESUM - the first dedicated multi-document summarization dataset in Hebrew, showing the phenomenon is common across seven diverse languages.

Conclusion: Historical newspaper front-page teasers provide a valuable source for creating summarization datasets in under-represented languages, enabling multi-document summarization research.

Abstract: High quality summarization data remains scarce in under-represented languages. However, historical newspapers, made available through recent digitization efforts, offer an abundant source of untapped, naturally annotated data. In this work, we present a novel method for collecting naturally occurring summaries via Front-Page Teasers, where editors summarize full length articles. We show that this phenomenon is common across seven diverse languages and supports multi-document summarization. To scale data collection, we develop an automatic process, suited to varying linguistic resource levels. Finally, we apply this process to a Hebrew newspaper title, producing HEBTEASESUM, the first dedicated multi-document summarization dataset in Hebrew.

Method: Used electronic health record data to cluster AKI patients into clinical states based on longitudinal medical codes and creatinine measurements, then applied multi-state modeling to estimate transition probabilities and CKD progression.

Result: Of 20,699 AKI patients, 3,491 (17%) developed CKD. Identified 15 distinct post-AKI states with different CKD probabilities, with most patients (75%) remaining stable or making only one transition. Found both established and novel CKD risk factors with varying impacts across states.

Conclusion: The study demonstrates a data-driven approach for identifying high-risk AKI patients, supporting development of decision-support tools for early CKD detection and intervention.

Abstract: Patients with acute kidney injury (AKI) are at high risk of developing chronic kidney disease (CKD), but identifying those at greatest risk remains challenging. We used electronic health record (EHR) data to dynamically track AKI patients’ clinical evolution and characterize AKI-to-CKD progression. Post-AKI clinical states were identified by clustering patient vectors derived from longitudinal medical codes and creatinine measurements. Transition probabilities between states and progression to CKD were estimated using multi-state modeling. After identifying common post-AKI trajectories, CKD risk factors in AKI subpopulations were identified through survival analysis. Of 20,699 patients with AKI at admission, 3,491 (17%) developed CKD. We identified fifteen distinct post-AKI states, each with different probabilities of CKD development. Most patients (75%, n=15,607) remained in a single state or made only one transition during the study period. Both established (e.g., AKI severity, diabetes, hypertension, heart failure, liver disease) and novel CKD risk factors, with their impact varying across these clinical states. This study demonstrates a data-driven approach for identifying high-risk AKI patients, supporting the development of decision-support tools for early CKD detection and intervention.

Method: Constructed manually annotated dataset of news articles, assessed annotation consistency, bias polarity, and inter-model agreement using multiple LLMs including GPT, BERT, RoBERTa, and FLAN.

Result: RoBERTa achieved highest alignment with human labels among traditional transformer models, while GPT demonstrated strongest overall agreement in zero-shot setting. Fine-tuned RoBERTa obtained highest accuracy and strongest human alignment.

Conclusion: Systematic differences exist in how humans and LLMs perceive political slant, highlighting need for hybrid evaluation frameworks combining human interpretability with model scalability.

Abstract: Detecting political bias in news media is a complex task that requires interpreting subtle linguistic and contextual cues. Although recent advances in Natural Language Processing (NLP) have enabled automatic bias classification, the extent to which large language models (LLMs) align with human judgment still remains relatively underexplored and not yet well understood. This study aims to present a comparative framework for evaluating the detection of political bias across human annotations and multiple LLMs, including GPT, BERT, RoBERTa, and FLAN. We construct a manually annotated dataset of news articles and assess annotation consistency, bias polarity, and inter-model agreement to quantify divergence between human and model perceptions of bias. Experimental results show that among traditional transformer-based models, RoBERTa achieves the highest alignment with human labels, whereas generative models such as GPT demonstrate the strongest overall agreement with human annotations in a zero-shot setting. Among all transformer-based baselines, our fine-tuned RoBERTa model acquired the highest accuracy and the strongest alignment with human-annotated labels. Our findings highlight systematic differences in how humans and LLMs perceive political slant, underscoring the need for hybrid evaluation frameworks that combine human interpretability with model scalability in automated media bias detection.

Method: Developed via staged instruction tuning, chain of thought learning, and graph-grounded retrieval using a large domain-specialized clinical corpus and clinician-validated reasoning set.

Result: Achieves state-of-the-art accuracy across multicenter EHR narratives and public benchmarks, with robust generalization and stability under noisy/overlapping phenotypes. Human studies show performance on par with experienced physicians.

Conclusion: Advances a narrative-first, knowledge-integrated reasoning paradigm that shortens diagnostic odyssey and enables auditable, clinically translatable decision support.

Abstract: Rare diseases affect hundreds of millions worldwide, yet diagnosis often spans years. Convectional pipelines decouple noisy evidence extraction from downstream inferential diagnosis, and general/medical large language models (LLMs) face scarce real world electronic health records (EHRs), stale domain knowledge, and hallucinations. We assemble a large, domain specialized clinical corpus and a clinician validated reasoning set, and develop RareSeek R1 via staged instruction tuning, chain of thought learning, and graph grounded retrieval. Across multicenter EHR narratives and public benchmarks, RareSeek R1 attains state of the art accuracy, robust generalization, and stability under noisy or overlapping phenotypes. Augmented retrieval yields the largest gains when narratives pair with prioritized variants by resolving ambiguity and aligning candidates to mechanisms. Human studies show performance on par with experienced physicians and consistent gains in assistive use. Notably, transparent reasoning highlights decisive non phenotypic evidence (median 23.1%, such as imaging, interventions, functional tests) underpinning many correct diagnoses. This work advances a narrative first, knowledge integrated reasoning paradigm that shortens the diagnostic odyssey and enables auditable, clinically translatable decision support.

Method: Developed a quantitative model of posterior probability for plausible interpretations by synthesizing statistical language models with human behavioral data, going beyond prior narrow evaluations of alternative interpretations.

Result: The model successfully explains fine gradations in comparative illusion strength and the previously unexplained effect of pronominal vs. full noun phrase than-clause subjects, with empirical validation.

Conclusion: Findings support noisy-channel inference as a unified computational-level theory for diverse language processing phenomena, including both illusory and non-illusory contexts.

Abstract: Like visual processing, language processing is susceptible to illusions in which people systematically misperceive stimuli. In one such case–the comparative illusion (CI), e.g., More students have been to Russia than I have–comprehenders tend to judge the sentence as acceptable despite its underlying nonsensical comparison. Prior research has argued that this phenomenon can be explained as Bayesian inference over a noisy channel: the posterior probability of an interpretation of a sentence is proportional to both the prior probability of that interpretation and the likelihood of corruption into the observed (CI) sentence. Initial behavioral work has supported this claim by evaluating a narrow set of alternative interpretations of CI sentences and showing that comprehenders favor interpretations that are more likely to have been corrupted into the illusory sentence. In this study, we replicate and go substantially beyond this earlier work by directly predicting the strength of illusion with a quantitative model of the posterior probability of plausible interpretations, which we derive through a novel synthesis of statistical language models with human behavioral data. Our model explains not only the fine gradations in the strength of CI effects, but also a previously unexplained effect caused by pronominal vs. full noun phrase than-clause subjects. These findings support a noisy-channel theory of sentence comprehension by demonstrating that the theory makes novel predictions about the comparative illusion that bear out empirically. This outcome joins related evidence of noisy channel processing in both illusory and non-illusory contexts to support noisy channel inference as a unified computational-level theory of diverse language processing phenomena.

Method: Proposes a comprehensive framework distinguishing instruction bias, annotator bias, and contextual/cultural bias; reviews detection methods (inter-annotator agreement, model disagreement, metadata analysis); outlines mitigation strategies including diverse annotator recruitment and iterative guideline refinement.

Result: Develops a typology of annotation bias, synthesizes detection metrics, proposes ensemble-based bias mitigation for multilingual settings, and provides ethical analysis of annotation processes.

Conclusion: The framework aims to inform more equitable and culturally grounded annotation pipelines for LLMs, addressing critical challenges in multilingual NLP dataset development.

Abstract: Annotation bias in NLP datasets remains a major challenge for developing multilingual Large Language Models (LLMs), particularly in culturally diverse settings. Bias from task framing, annotator subjectivity, and cultural mismatches can distort model outputs and exacerbate social harms. We propose a comprehensive framework for understanding annotation bias, distinguishing among instruction bias, annotator bias, and contextual and cultural bias. We review detection methods (including inter-annotator agreement, model disagreement, and metadata analysis) and highlight emerging techniques such as multilingual model divergence and cultural inference. We further outline proactive and reactive mitigation strategies, including diverse annotator recruitment, iterative guideline refinement, and post-hoc model adjustments. Our contributions include: (1) a typology of annotation bias; (2) a synthesis of detection metrics; (3) an ensemble-based bias mitigation approach adapted for multilingual settings, and (4) an ethical analysis of annotation processes. Together, these insights aim to inform more equitable and culturally grounded annotation pipelines for LLMs.

Method: Retrieval-augmented generation pipeline with synthetic data generation, semantic clause retrieval, acceptability classification, and reward-based alignment.

Result: Achieves over 80% accuracy in identifying and optimizing problematic revisions under real-world, low-resource conditions.

Conclusion: The framework offers a practical solution for accelerating contract revision workflows while maintaining high accuracy in challenging data environments.

Abstract: Contract management involves reviewing and negotiating provisions, individual clauses that define rights, obligations, and terms of agreement. During this process, revisions to provisions are proposed and iteratively refined, some of which may be problematic or unacceptable. Automating this workflow is challenging due to the scarcity of labeled data and the abundance of unstructured legacy contracts. In this paper, we present a modular framework designed to streamline contract management through a retrieval-augmented generation (RAG) pipeline. Our system integrates synthetic data generation, semantic clause retrieval, acceptability classification, and reward-based alignment to flag problematic revisions and generate improved alternatives. Developed and evaluated in collaboration with an industry partner, our system achieves over 80% accuracy in both identifying and optimizing problematic revisions, demonstrating strong performance under real-world, low-resource conditions and offering a practical means of accelerating contract revision workflows.

Method: Analyzed 20 English novels/writings using regression analysis of log(type)-log(token) data with linear and quadratic terms, and used a random drawing colored ball model to explain the curvature.

Result: Found consistent linear coefficient slightly larger than 1 and quadratic coefficient around -0.02, with perfect fit using quadratic functions. The curvature corresponds to negative pseudo-variance in the random drawing model.

Conclusion: The type-token relationship is better modeled by quadratic functions in log-log scale rather than simple power laws, providing a more accurate characterization of vocabulary growth in texts.

Abstract: Heaps’ or Herdan’s law characterizes the word-type vs. word-token relation by a power-law function, which is concave in linear-linear scale but a straight line in log-log scale. However, it has been observed that even in log-log scale, the type-token curve is still slightly concave, invalidating the power-law relation. At the next-order approximation, we have shown, by twenty English novels or writings (some are translated from another language to English), that quadratic functions in log-log scale fit the type-token data perfectly. Regression analyses of log(type)-log(token) data with both a linear and quadratic term consistently lead to a linear coefficient of slightly larger than 1, and a quadratic coefficient around -0.02. Using the random drawing colored ball from the bag with replacement" model, we have shown that the curvature of the log-log scale is identical to a pseudo-variance" which is negative. Although a pseudo-variance calculation may encounter numeric instability when the number of tokens is large, due to the large values of pseudo-weights, this formalism provides a rough estimation of the curvature when the number of tokens is small.

Method: Formulates student reasoning as multi-step sequential decision problem, uses MCTS to explore correction paths, leverages BFS and final-answer evaluation for reward signals, and distributes rewards via back-propagation for process supervision.

Result: Significantly outperforms existing methods on ProcessBench, MR-GSM8K, and their MSEB benchmark in terms of effectiveness and overall performance.

Conclusion: SMRC provides an effective approach for educational-style mathematical reasoning correction that aligns with student thinking processes and offers comprehensive evaluation through solution accuracy and correct-step retention metrics.

Abstract: Large language models (LLMs) often make reasoning errors when solving mathematical problems, and how to automatically detect and correct these errors has become an important research direction. However, existing approaches \textit{mainly focus on self-correction within the model}, which falls short of the teacher-style correction required in educational settings, \textit{i.e.}, systematically guiding and revising a student’s problem-solving process. To address this gap, we propose \texttt{SMRC} (\textit{\underline{S}tudent \underline{M}athematical \underline{R}easoning \underline{C}orrection}), a novel method that aligns LLMs with student reasoning. Specifically, \texttt{SMRC} formulates student reasoning as a multi-step sequential decision problem and introduces Monte Carlo Tree Search (MCTS) to explore optimal correction paths. To reduce the cost of the annotating process-level rewards, we leverage breadth-first search (BFS) guided by LLMs and final-answer evaluation to generate reward signals, which are then distributed across intermediate reasoning steps via a back-propagation mechanism, enabling fine-grained process supervision. Additionally, we construct a benchmark for high school mathematics, MSEB (Multi-Solution Error Benchmark), consisting of 158 instances that include problem statements, student solutions, and correct reasoning steps. We further propose a dual evaluation protocol centered on \textbf{solution accuracy} and \textbf{correct-step retention}, offering a comprehensive measure of educational applicability. Experiments demonstrate that \texttt{SMRC} significantly outperforms existing methods on two public datasets (ProcessBench and MR-GSM8K) and our MSEB in terms of effectiveness and overall performance. The code and data are available at https://github.com/Mind-Lab-ECNU/SMRC.

Method: Using sparse autoencoders to extract interpretable features from text, investigating the role of prompting and language aspects in encoding physics.

Result: LLMs demonstrate the ability to encode physical information from scientific measurements through their embeddings.

Conclusion: LLMs can effectively codify physical summary statistics, with prompting and specific language aspects playing crucial roles in how physics is encoded.

Abstract: Large Language Models have demonstrated the ability to generalize well at many levels across domains, modalities, and even shown in-context learning capabilities. This enables research questions regarding how they can be used to encode physical information that is usually only available from scientific measurements, and loosely encoded in textual descriptions. Using astrophysics as a test bed, we investigate if LLM embeddings can codify physical summary statistics that are obtained from scientific measurements through two main questions: 1) Does prompting play a role on how those quantities are codified by the LLM? and 2) What aspects of language are most important in encoding the physics represented by the measurement? We investigate this using sparse autoencoders that extract interpretable features from the text.

Method: Leveraged large language models to create synthetic ground-truth annotations for historical texts, then used this data to fine-tune spaCy models for domain-specific NLP tasks.

Result: Achieved significant performance gains on period-specific tests for part-of-speech annotations, lemmatization, and named entity recognition using relatively limited amounts of synthetic data.

Conclusion: Domain-specific models are crucial for historical NLP, and even small amounts of LLM-generated synthetic data can substantially improve NLP tools for under-resourced corpora in computational humanities.

Abstract: Historical and low-resource NLP remains challenging due to limited annotated data and domain mismatches with modern, web-sourced corpora. This paper outlines our work in using large language models (LLMs) to create ground-truth annotations for historical French (16th-20th centuries) and Chinese (1900-1950) texts. By leveraging LLM-generated ground truth on a subset of our corpus, we were able to fine-tune spaCy to achieve significant gains on period-specific tests for part-of-speech (POS) annotations, lemmatization, and named entity recognition (NER). Our results underscore the importance of domain-specific models and demonstrate that even relatively limited amounts of synthetic data can improve NLP tools for under-resourced corpora in computational humanities research.

Method: Uses VALOR framework with multi-expert reasoning using large generative models with Chain-of-Thought prompting, computes semantic alignment score between modalities, and employs meta-fusion strategy for final classification.

Result: VALOR consistently outperforms baseline models on curated multimodal complaint dataset, especially in complex scenarios where information is distributed across text and images.

Conclusion: Multimodal interaction and expert validation are valuable for practical complaint understanding systems, supporting UN SDGs 9 and 12 through improved service infrastructure and responsible consumption.

Abstract: Existing approaches to complaint analysis largely rely on unimodal, short-form content such as tweets or product reviews. This work advances the field by leveraging multimodal, multi-turn customer support dialogues, where users often share both textual complaints and visual evidence (e.g., screenshots, product photos) to enable fine-grained classification of complaint aspects and severity. We introduce VALOR, a Validation-Aware Learner with Expert Routing, tailored for this multimodal setting. It employs a multi-expert reasoning setup using large-scale generative models with Chain-of-Thought (CoT) prompting for nuanced decision-making. To ensure coherence between modalities, a semantic alignment score is computed and integrated into the final classification through a meta-fusion strategy. In alignment with the United Nations Sustainable Development Goals (UN SDGs), the proposed framework supports SDG 9 (Industry, Innovation and Infrastructure) by advancing AI-driven tools for robust, scalable, and context-aware service infrastructure. Further, by enabling structured analysis of complaint narratives and visual context, it contributes to SDG 12 (Responsible Consumption and Production) by promoting more responsive product design and improved accountability in consumer services. We evaluate VALOR on a curated multimodal complaint dataset annotated with fine-grained aspect and severity labels, showing that it consistently outperforms baseline models, especially in complex complaint scenarios where information is distributed across text and images. This study underscores the value of multimodal interaction and expert validation in practical complaint understanding systems. Resources related to data and codes are available here: https://github.com/sarmistha-D/VALOR

Method: Developed a BiLSTM-CRF morphological segmenter using bootstrapped training from minimal manual annotation, compared word-level, morpheme-based, and BPE tokenization approaches, and evaluated Word2Vec embeddings using comprehensive metrics including similarity preservation, clustering quality, and semantic organization.

Result: BPE evaluated only 28.6% of test cases compared to 68.7% for morpheme model, creating artificial performance inflation. When assessed comprehensively, morpheme-based tokenization demonstrated superior embedding space organization, better semantic neighborhood structure, and more balanced coverage across morphological complexity levels.

Conclusion: Coverage-aware evaluation is crucial in low-resource language processing, and morpheme-based tokenization offers better overall performance despite BPE’s apparent advantages in limited test scenarios.

Abstract: We investigate tokenization strategies for Kurdish word embeddings by comparing word-level, morpheme-based, and BPE approaches on morphological similarity preservation tasks. We develop a BiLSTM-CRF morphological segmenter using bootstrapped training from minimal manual annotation and evaluate Word2Vec embeddings across comprehensive metrics including similarity preservation, clustering quality, and semantic organization. Our analysis reveals critical evaluation biases in tokenization comparison. While BPE initially appears superior in morphological similarity, it evaluates only 28.6% of test cases compared to 68.7% for morpheme model, creating artificial performance inflation. When assessed comprehensively, morpheme-based tokenization demonstrates superior embedding space organization, better semantic neighborhood structure, and more balanced coverage across morphological complexity levels. These findings highlight the importance of coverage-aware evaluation in low-resource language processing and offers different tokenization methods for low-resourced language processing.

Method: Extensive analysis of scientific literature, patent databases, and experimental data using LLMs to enable more flexible and informed R&D workflows.

Result: LLMs dramatically improve research process efficiency and effectiveness, accelerating innovation cycles and reducing time-to-market for breakthrough ideas.

Conclusion: Large Language Models serve as powerful tools that transform R&D processes through multiple functions including knowledge automation, hypothesis generation, transdisciplinary integration, and ecosystem collaboration.

Abstract: This study analyzes the multiple functions of Large Language Models (LLMs) in transforming research and development (R&D) processes. By automating knowledge discovery, boosting hypothesis creation, integrating transdisciplinary insights, and enabling cooperation within innovation ecosystems, LLMs dramatically improve the efficiency and effectiveness of research processes. Through extensive analysis of scientific literature, patent databases, and experimental data, these models enable more flexible and informed R&D workflows, ultimately accelerating innovation cycles and lowering time-to-market for breakthrough ideas.

Method: Constructed a comprehensive Japanese text dataset comprising 4,672 domestic and 9,607 overseas travelogues, totaling over 31 million words.

Result: Successfully created and released the NAIST ATD dataset free of charge for academic research, providing a standardized resource for the research community.

Conclusion: The dataset enables researchers to conduct investigations on the same data, ensuring transparency and reproducibility in travelogue-related research studies.

Abstract: We have constructed NAIST Academic Travelogue Dataset (ATD) and released it free of charge for academic research. This dataset is a Japanese text dataset with a total of over 31 million words, comprising 4,672 Japanese domestic travelogues and 9,607 overseas travelogues. Before providing our dataset, there was a scarcity of widely available travelogue data for research purposes, and each researcher had to prepare their own data. This hinders the replication of existing studies and fair comparative analysis of experimental results. Our dataset enables any researchers to conduct investigation on the same data and to ensure transparency and reproducibility in research. In this paper, we describe the academic significance, characteristics, and prospects of our dataset.

Method: Used simulations to study codes constrained by predictive information, and analyzed crosslinguistic text corpora to compare predictive information in actual human languages against baselines at phonological, morphological, syntactic, and lexical semantic levels.

Result: Simulations showed that predictive information-constrained codes break messages into word-like and phrase-like groups with systematic, local expression. Crosslinguistic analysis revealed human languages reduce predictive information compared to baselines across all linguistic levels studied.

Conclusion: Human language’s systematic structure emerges from constraints on predictive information, reinforcing that linguistic structures are shaped by communication under general cognitive constraints rather than being arbitrary.

Abstract: Human language has a distinct systematic structure, where utterances break into individually meaningful words which are combined to form phrases. We show that natural-language-like systematicity arises in codes that are constrained by a statistical measure of complexity called predictive information, also known as excess entropy. Predictive information is the mutual information between the past and future of a stochastic process. In simulations, we find that such codes break messages into groups of approximately independent features which are expressed systematically and locally, corresponding to words and phrases. Next, drawing on crosslinguistic text corpora, we find that actual human languages are structured in a way that reduces predictive information compared to baselines at the levels of phonology, morphology, syntax, and lexical semantics. Our results establish a link between the statistical and algebraic structure of language and reinforce the idea that these structures are shaped by communication under general cognitive constraints.

Method: Collected a new dataset with 12,000+ annotations from 124 annotators, including native language and English proficiency information, to analyze LLM performance across different user profiles.

Result: Performance discrepancies occur between native and non-native English speakers, persist when comparing Western native speakers with others, and show strong anchoring effect that degrades response quality for non-native speakers.

Conclusion: LLMs exhibit demographic biases in response quality, with non-native English speakers receiving inferior responses, highlighting the need for more equitable AI systems.

Abstract: Large Language Models (LLMs) excel at providing information acquired during pretraining on large-scale corpora and following instructions through user prompts. This study investigates whether the quality of LLM responses varies depending on the demographic profile of users. Considering English as the global lingua franca, along with the diversity of its dialects among speakers of different native languages, we explore whether non-native English speakers receive lower-quality or even factually incorrect responses from LLMs more frequently. Our results show that performance discrepancies occur when LLMs are prompted by native versus non-native English speakers and persist when comparing native speakers from Western countries with others. Additionally, we find a strong anchoring effect when the model recognizes or is made aware of the user’s nativeness, which further degrades the response quality when interacting with non-native speakers. Our analysis is based on a newly collected dataset with over 12,000 unique annotations from 124 annotators, including information on their native language and English proficiency.

Method: Evaluate several methods for grounded prompt perturbation, generating perturbations and filtering based on similarity to text and/or modality data. Then retrain models on augmented data created through these perturbations.

Result: After retraining on augmented data, models show improved accuracy and more stable performance on perturbed test data across all perturbation conditions. Error analysis reveals consistent performance improvement patterns across domains.

Conclusion: Data augmentation with prompt perturbations helps multimodal foundation models handle domain shifts more effectively and improves general reasoning capabilities, making them more robust to text input variations.

Abstract: Multimodal foundation models (MFMs) such as OFASys show the potential to unlock analysis of complex data such as images, videos, and audio data via text prompts alone. However, their performance may suffer in the face of text input that differs even slightly from their training distribution, which is surprising considering the use of modality-specific data to “ground” the text input. This study demonstrates that prompt instability is a major concern for MFMs, leading to a consistent drop in performance across all modalities, but that instability can be mitigated with additional training with augmented data. We evaluate several methods for grounded prompt perturbation, where we generate perturbations and filter based on similarity to text and/or modality data. After re-training the models on the augmented data, we find improved accuracy and more stable performance on the perturbed test data regardless of perturbation condition, suggesting that the data augmentation strategy helps the models handle domain shifts more effectively. In error analysis, we find consistent patterns of performance improvement across domains, suggesting that retraining on prompt perturbations tends to help general reasoning capabilities in MFMs.

Method: Rigorous testing across diverse domains including computer science, mathematics, natural sciences, medicine, linguistics, and social sciences through various challenging tasks and benchmarks.

Result: o1-preview demonstrated remarkable performance: 83.3% success in competitive programming, superior radiology report generation, 100% accuracy in high school math, advanced natural language inference, impressive chip design capabilities, and strong performance in specialized fields like anthropology and quantitative investing.

Conclusion: The model shows significant progress toward artificial general intelligence, excelling in intricate reasoning and knowledge integration across fields, though some limitations remain with simpler problems and highly specialized concepts.

Abstract: This comprehensive study evaluates the performance of OpenAI’s o1-preview large language model across a diverse array of complex reasoning tasks, spanning multiple domains, including computer science, mathematics, natural sciences, medicine, linguistics, and social sciences. Through rigorous testing, o1-preview demonstrated remarkable capabilities, often achieving human-level or superior performance in areas ranging from coding challenges to scientific reasoning and from language processing to creative problem-solving. Key findings include: -83.3% success rate in solving complex competitive programming problems, surpassing many human experts. -Superior ability in generating coherent and accurate radiology reports, outperforming other evaluated models. -100% accuracy in high school-level mathematical reasoning tasks, providing detailed step-by-step solutions. -Advanced natural language inference capabilities across general and specialized domains like medicine. -Impressive performance in chip design tasks, outperforming specialized models in areas such as EDA script generation and bug analysis. -Remarkable proficiency in anthropology and geology, demonstrating deep understanding and reasoning in these specialized fields. -Strong capabilities in quantitative investing. O1 has comprehensive financial knowledge and statistical modeling skills. -Effective performance in social media analysis, including sentiment analysis and emotion recognition. The model excelled particularly in tasks requiring intricate reasoning and knowledge integration across various fields. While some limitations were observed, including occasional errors on simpler problems and challenges with certain highly specialized concepts, the overall results indicate significant progress towards artificial general intelligence.

Method: Measured speech-to-text performance on Italian dialects, analyzed correlation with linguistic similarity to highest-performing variety, used dialectometry methods, and applied geostatistical methods to predict zero-shot performance.

Result: Found substantial geographical performance disparity (-0.5 correlation with linguistic similarity to highest-performing dialect), indicating bias towards dialects more similar to standard variety. Geographical information significantly improved prediction of zero-shot performance.

Conclusion: Dialect variation within categories critically affects NLP performance, with geographical structure in performance distribution and bias towards standard-like dialects.

Abstract: There is increasing interest in looking at dialects in NLP. However, most work to date still treats dialects as discrete categories. For instance, evaluative work in variation-oriented NLP for English often works with Indian English or African-American Venacular English as homogeneous categories (Faisal et al., 2024; Ziems et al., 2023), yet even within one variety there is substantial variation. We examine within-dialect variation and show that performance critically varies within categories. We measure speech-to-text performance on Italian dialects, and empirically observe a geographical performance disparity. This disparity correlates substantially (-0.5) with linguistic similarity to the highest performing dialect variety. We cross-examine our results against dialectometry methods, and interpret the performance disparity to be due to a bias towards dialects that are more similar to the standard variety in the speech-to-text model examined. We additionally leverage geostatistical methods to predict zero-shot performance at unseen sites, and find the incorporation of geographical information to substantially improve prediction performance, indicating there to be geographical structure in the performance distribution.

Method: Used dictator games to investigate how different personas and experimental framings affect LLM agents’ altruistic behavior, comparing behaviors within and across LLM families and with human behaviors.

Result: LLM agents’ reasoning doesn’t consistently show textual markers of human decision-making, and their alignment with human behavior varies substantially across model architectures and prompt formulations without clear patterns.

Conclusion: Prosocial AI is an urgent research direction as society integrates machine intelligence, but current LLM agents don’t reliably exhibit human-like prosocial behaviors.

Abstract: As Large Language Model (LLM)-based agents increasingly engage with human society, how well do we understand their prosocial behaviors? We (1) investigate how LLM agents’ prosocial behaviors can be induced by different personas and benchmarked against human behaviors; and (2) introduce a social science approach to evaluate LLM agents’ decision-making. We explored how different personas and experimental framings affect these AI agents’ altruistic behavior in dictator games and compared their behaviors within the same LLM family, across various families, and with human behaviors. The findings reveal that merely assigning a human-like identity to LLMs does not produce human-like behaviors. These findings suggest that LLM agents’ reasoning does not consistently exhibit textual markers of human decision-making in dictator games and that their alignment with human behavior varies substantially across model architectures and prompt formulations; even worse, such dependence does not follow a clear pattern. As society increasingly integrates machine intelligence, “Prosocial AI” emerges as a promising and urgent research direction in philanthropic studies.

Method: Uses advanced data preprocessing, transformer-based models via Hugging Face framework, and focuses on model fine-tuning and robustness.

Result: Provides insights into effective AI deployment with emphasis on handling multilingual data, reducing bias, and ensuring model reliability.

Conclusion: The work contributes to developing ethically sound and effective AI solutions through improved NLP techniques and responsible model practices.

Abstract: With a focus on natural language processing (NLP) and the role of large language models (LLMs), we explore the intersection of machine learning, deep learning, and artificial intelligence. As artificial intelligence continues to revolutionize fields from healthcare to finance, NLP techniques such as tokenization, text classification, and entity recognition are essential for processing and understanding human language. This paper discusses advanced data preprocessing techniques and the use of frameworks like Hugging Face for implementing transformer-based models. Additionally, it highlights challenges such as handling multilingual data, reducing bias, and ensuring model robustness. By addressing key aspects of data processing and model fine-tuning, this work aims to provide insights into deploying effective and ethically sound AI solutions.

Method: Scientometric and thematic analysis of 9836 unique articles from WoS, Scopus, and Lens databases, including cluster analysis, keyword bursts, centrality measures, and detailed review of 18 purposefully selected articles.

Result: AI development directly correlates with increased contributions to translation industry, with neural networking algorithms and deep language learning models being key drivers. Key hotspots identified include machine translation, low-resource languages, and neural machine translation.

Conclusion: While AI has significantly advanced translation capabilities through neural networks and large language models, substantial research is still needed to address challenges with low-resource, multi-dialectical languages and cultural/religious registers.

Abstract: This study provides a comprehensive analysis of artificial intelligence (AI) contribution to research in the translation industry (ACTI), synthesizing it over forty-five years from 1980-2024. 13220 articles were retrieved from three sources, namely WoS, Scopus, and Lens; 9836 were unique records, which were used for the analysis. We provided two types of analysis, viz., scientometric and thematic, focusing on Cluster, Subject categories, Keywords, Bursts, Centrality and Research Centers as for the former. For the latter, we provided a thematic review for 18 articles, selected purposefully from the articles involved, centering on purpose, approach, findings, and contribution to ACTI future directions. This study is significant for its valuable contribution to ACTI knowledge production over 45 years, emphasizing several trending issues and hotspots including Machine translation, Statistical machine translation, Low-resource language, Large language model, Arabic dialects, Translation quality, and Neural machine translation. The findings reveal that the more AI develops, the more it contributes to translation industry, as Neural Networking Algorithms have been incorporated and Deep Language Learning Models like ChatGPT have been launched. However, much rigorous research is still needed to overcome several problems encountering translation industry, specifically concerning low-resource, multi-dialectical and free word order languages, and cultural and religious registers.

Method: Constructed by synthesizing long-context reasoning questions from short-context ones through context expansion, resulting in 794 multiple-choice questions across three task categories.

Result: Evaluation of 21 LLMs shows significant performance drops as context length increases, with even state-of-the-art models having substantial room for improvement in robust reasoning.

Conclusion: LongReason enables comprehensive evaluation of long-context reasoning capabilities and reveals current LLMs’ limitations in maintaining reasoning performance across extended contexts.

Abstract: Large language models (LLMs) have demonstrated remarkable progress in understanding long-context inputs. However, benchmarks for evaluating the long-context reasoning abilities of LLMs fall behind the pace. Existing benchmarks often focus on a narrow range of tasks or those that do not demand complex reasoning. To address this gap and enable a more comprehensive evaluation of the long-context reasoning capabilities of current LLMs, we propose a new synthetic benchmark, LongReason, which is constructed by synthesizing long-context reasoning questions from a varied set of short-context reasoning questions through context expansion. LongReason consists of 794 multiple-choice reasoning questions with diverse reasoning patterns across three task categories: reading comprehension, logical inference, and mathematical word problems. We evaluate 21 LLMs on LongReason, revealing that most models experience significant performance drops as context length increases. Our further analysis shows that even state-of-the-art LLMs still have significant room for improvement in providing robust reasoning across different tasks. We have open-sourced LongReason under https://huggingface.co/datasets/lz1bytedance/LongReason to support the comprehensive evaluation of LLMs’ long-context reasoning capabilities.

Method: MoM uses multiple independent memory states with a router network that directs input tokens to specific memory states, serving as a general framework compatible with various memory update mechanisms in linear models.

Result: MoM outperforms existing linear sequence models on downstream language tasks, especially recall-intensive tasks, and achieves performance comparable to Transformer models while maintaining linear complexity during training and constant complexity during inference.

Conclusion: MoM effectively addresses the memory capacity limitations of linear sequence models through multiple independent memory states, enabling superior performance on recall-intensive tasks while preserving computational efficiency advantages.

Abstract: Linear sequence modeling methods, such as linear attention, state space modeling, and linear RNNs, offer significant efficiency improvements by reducing the complexity of training and inference. However, these methods typically compress the entire input sequence into a single fixed-size memory state, which leads to suboptimal performance on recall-intensive tasks. To address this limitation, we introduce a novel architecture called Mixture-of-Memories (MoM). MoM utilizes multiple independent memory states, with a router network directing input tokens to specific memory states. This approach greatly enhances the overall memory capacity while minimizing memory interference. MoM serves as a general framework that can be seamlessly combined with diverse memory update mechanisms across linear models. As a result, MoM performs exceptionally well on recall-intensive tasks, surpassing existing linear sequence modeling techniques. Despite incorporating multiple memory states, the computation of each memory state remains linear in complexity, allowing MoM to retain the linear-complexity advantage during training, while constant-complexity during inference. Our experimental results show that MoM outperforms current linear sequence models on downstream language tasks, particularly recall-intensive tasks, and even achieves performance comparable to Transformer models. The code is released at https://github.com/OpenSparseLLMs/MoM and is also released as a part of https://github.com/OpenSparseLLMs/Linear-MoE.

Method: Developed as a two-stage ensemble of larger open-weight LLMs or as a small fine-tuned evaluation model, using error spans for explanatory feedback and ensuring generalizability to unseen tasks, domains, and aspects.

Result: Achieves competitive correlation with human judgments, outperforms state-of-the-art models on certain tasks, provides explanations more than twice as accurate, and maintains full reproducibility.

Conclusion: OpeNLGauge successfully addresses limitations of existing LLM-based metrics by offering an open-source, reference-free solution with accurate explanatory feedback and competitive performance.

Abstract: Large Language Models (LLMs) have demonstrated great potential as evaluators of NLG systems, allowing for high-quality, reference-free, and multi-aspect assessments. However, existing LLM-based metrics suffer from two major drawbacks: reliance on proprietary models to generate training data or perform evaluations, and a lack of fine-grained, explanatory feedback. In this paper, we introduce OpeNLGauge, a fully open-source, reference-free NLG evaluation metric that provides accurate explanations based on error spans. OpeNLGauge is available as a two-stage ensemble of larger open-weight LLMs, or as a small fine-tuned evaluation model, with confirmed generalizability to unseen tasks, domains and aspects. Our extensive meta-evaluation shows that OpeNLGauge achieves competitive correlation with human judgments, outperforming state-of-the-art models on certain tasks while maintaining full reproducibility and providing explanations more than twice as accurate.

Method: Proposes ACoRN with two training steps: 1) offline data augmentation to enhance robustness against retrieval noise, and 2) fine-tuning to generate summaries centered around key information supporting correct answers, addressing positional bias and multi-document utilization issues.

Result: T5-large trained with ACoRN improves EM and F1 scores while preserving answer strings as direct evidence, particularly excelling on datasets with many accuracy-reducing documents.

Conclusion: ACoRN effectively addresses noise in retrieved documents and improves compression quality in real-world RAG scenarios, making it highly useful for practical applications.

Abstract: Abstractive compression utilizes smaller langauge models to condense query-relevant context, reducing computational costs in retrieval-augmented generation (RAG). However,retrieved documents often include information that is either irrelevant to answering the query or misleading due to factual incorrect content, despite having high relevance scores. This behavior indicates that abstractive compressors are more likely to omit important information essential for the correct answer, especially in long contexts where attention dispersion occurs. To address this issue, we categorize retrieved documents in a more fine-grained manner and propose Abstractive Compression Robust against Noise (ACoRN), which introduces two novel training steps. First, we use offline data augmentation on the training dataset to enhance compressor robustness against two distinct types of retrieval noise. Second, since the language modelbased compressor cannot fully utilize information from multiple retrieved documents and exhibits positional bias, we perform finetuning to generate summaries centered around key information that directly supports the correct answer. Our experiments demonstrate that T5-large, trained with ACoRN as a compressor, improves EM and F1 scores while preserving the answer string, which could serve as direct evidence. ACoRN excels on datasets with many accuracy-reducing documents, making it highly useful in real-world scenarios.

Method: PromptObfus uses anti-adversarial learning by framing prompt desensitization as a masked language modeling task. It replaces privacy-sensitive terms with [MASK] tokens, trains a desensitization model to generate candidate replacements, and selects candidates based on gradient feedback from a surrogate model to minimize task output disruption.

Result: The method was tested on three NLP tasks and effectively prevented privacy inference from remote LLMs while preserving task performance.

Conclusion: PromptObfus provides an effective solution for protecting privacy in LLM prompts by obfuscating sensitive information without compromising the utility of the prompts for their intended tasks.

Abstract: With the widespread use of LLMs, preserving privacy in user prompts has become crucial, as prompts risk exposing privacy and sensitive data to the cloud LLMs. Traditional techniques like homomorphic encryption, secure multi-party computation, and federated learning face challenges due to heavy computational costs and user participation requirements, limiting their applicability in LLM scenarios. In this paper, we propose PromptObfus, a novel method for desensitizing LLM prompts. The core idea of PromptObfus is “anti-adversarial” learning, which perturbs privacy words in the prompt to obscure sensitive information while retaining the stability of model predictions. Specifically, PromptObfus frames prompt desensitization as a masked language modeling task, replacing privacy-sensitive terms with a [MASK] token. A desensitization model is trained to generate candidate replacements for each masked position. These candidates are subsequently selected based on gradient feedback from a surrogate model, ensuring minimal disruption to the task output. We demonstrate the effectiveness of our approach on three NLP tasks. Results show that PromptObfus effectively prevents privacy inference from remote LLMs while preserving task performance.

Method: Uses speculative decoding scheme to split LLM workloads between edge and server GPUs, with proactive edge drafting to overlap token creation with verification, and pipeline-aware scheduling to interleave multiple user requests.

Result: Achieves 2.22x server throughput, 1.91x overall cost efficiency improvement, and 11.24% reduction in inter-token latency compared to server-only baseline.

Conclusion: SpecEdge introduces a scalable, cost-effective paradigm for LLM serving by effectively utilizing edge resources alongside server infrastructure.

Abstract: Large language models (LLMs) power many modern applications, but serving them at scale remains costly and resource-intensive. Current server-centric systems overlook consumer-grade GPUs at the edge. We introduce SpecEdge, an edge-assisted inference framework that splits LLM workloads between edge and server GPUs using a speculative decoding scheme, exchanging only token outputs over the network. SpecEdge employs proactive edge drafting to overlap edge token creation with server verification and pipeline-aware scheduling that interleaves multiple user requests to increase server-side throughput. Experiments show SpecEdge enhances overall cost efficiency by 1.91x through achieving 2.22x server throughput, and reduces inter token latency by 11.24% compared to a server-only baseline, introducing a scalable, cost-effective paradigm for LLM serving. The code is available at https://github.com/kaist-ina/specedge

Method: Used in-context learning with experiments on four diverse endangered languages, comparing probability-based vs instruction-based approaches, and evaluating on language modeling and ASR tasks.

Result: More relevant text samples improve performance; probability-based approach outperforms instruction-based; ICL enables LLMs to achieve ASR performance comparable to or better than dedicated language models.

Conclusion: LLMs can effectively learn unseen low-resource languages through in-context learning for speech recognition, offering a promising approach to expand language support without compromising existing capabilities.

Abstract: With approximately 7,000 languages spoken worldwide, current large language models (LLMs) support only a small subset. Prior research indicates LLMs can learn new languages for certain tasks without supervised data. We extend this investigation to speech recognition, investigating whether LLMs can learn unseen, low-resource languages through in-context learning (ICL). With experiments on four diverse endangered languages that LLMs have not been trained on, we find that providing more relevant text samples enhances performance in both language modelling and Automatic Speech Recognition (ASR) tasks. Furthermore, we show that the probability-based approach outperforms the traditional instruction-based approach in language learning. Lastly, we show ICL enables LLMs to achieve ASR performance that is comparable to or even surpasses dedicated language models trained specifically for these languages, while preserving the original capabilities of the LLMs. Our code is publicly available.

Method: MCTSr-Zero uses domain alignment to focus on conversational trajectories conforming to domain principles, plus regeneration and meta-prompt adaptation to broaden exploration of dialogue strategies.

Result: Fine-tuned PsyLLM model achieves state-of-the-art performance on PsyEval benchmark and other metrics for psychological counseling dialogues.

Conclusion: MCTSr-Zero effectively generates high-quality, principle-aligned conversational data for human-centric domains, addressing LLM challenges in adhering to complex psychological standards.

Abstract: The integration of Monte Carlo Tree Search (MCTS) with Large Language Models (LLMs) has demonstrated significant success in structured, problem-oriented tasks. However, applying these methods to open-ended dialogues, such as those in psychological counseling, presents unique challenges. Unlike tasks with objective correctness, success in therapeutic conversations depends on subjective factors like empathetic engagement, ethical adherence, and alignment with human preferences, for which strict “correctness” criteria are ill-defined. Existing result-oriented MCTS approaches can therefore produce misaligned responses. To address this, we introduce MCTSr-Zero, an MCTS framework designed for open-ended, human-centric dialogues. Its core innovation is “domain alignment”, which shifts the MCTS search objective from predefined end-states towards conversational trajectories that conform to target domain principles (e.g., empathy in counseling). Furthermore, MCTSr-Zero incorporates “Regeneration” and “Meta-Prompt Adaptation” mechanisms to substantially broaden exploration by allowing the MCTS to consider fundamentally different initial dialogue strategies. We evaluate MCTSr-Zero in psychological counseling by generating multi-turn dialogue data, which is used to fine-tune an LLM, PsyLLM. We also introduce PsyEval, a benchmark for assessing multi-turn psychological counseling dialogues. Experiments demonstrate that PsyLLM achieves state-of-the-art performance on PsyEval and other relevant metrics, validating MCTSr-Zero’s effectiveness in generating high-quality, principle-aligned conversational data for human-centric domains and addressing the LLM challenge of consistently adhering to complex psychological standards.

Method: Proposed Textual Stochastic Gradient Descent with Momentum (TSGD-M) that reweights updates through momentum sampling using bootstrapped minibatch validation accuracy as importance weights, with Gumbel-Top-k sampling for prompt generation.

Result: TSGD-M achieves consistent gains across 5 benchmarks and integrates seamlessly into existing prompt optimization frameworks like TextGrad, DSPy-COPRO, and AdalFlow.

Conclusion: The proposed TSGD-M method effectively addresses scalability challenges in prompt optimization by combining momentum-based sampling with efficient prompt generation techniques.

Abstract: LLM-based prompt optimization, that uses LLM-provided “textual gradients” (feedback) to refine prompts, has emerged an effective method for automatic prompt engineering. However, its scalability and stability are unclear when using more data in training. We systematically investigate the potential and challenges of scaling training data in textual gradient descent. We show that naively scaling training examples is infeasible due to both explicit context-length limits and an implicit context wall, where long-context degradation yields diminishing returns. Inspired by prior wisdom in stochastic gradient descent, we propose Textual Stochastic Gradient Descent with Momentum (TSGD-M), which reweights updates through momentum sampling, using bootstrapped minibatch validation accuracy as importance weights over historical prompts. We introduce Gumbel-Top-$k$ sampling for prompt generation, balancing exploration–exploitation and improving sampling efficiency while maintaining a low-variance running mean estimator. TSGD-M integrates seamlessly into existing prompt optimization frameworks, including TextGrad, DSPy-COPRO, and AdalFlow, and achieves consistent gains across 5 benchmarks.

Method: Proposes Generation after Recalibration (GenRecal) framework with a Recalibrator module that aligns and adapts feature representations between different VLM architectures, enabling cross-architecture knowledge distillation.

Result: Extensive experiments show GenRecal significantly improves baseline performances and eventually outperforms both large-scale open-source and closed-source VLMs on multiple challenging benchmarks.

Conclusion: GenRecal provides an effective solution for distilling knowledge across heterogeneous VLM architectures, overcoming the limitations of architecture-specific distillation methods and enabling more efficient VLM deployment.

Abstract: Recent advancements in vision-language models (VLMs) have leveraged large language models (LLMs) to achieve performance on par with closed-source systems like GPT-4V. However, deploying these models in real-world scenarios, particularly on resource-constrained devices, remains challenging due to their substantial computational demands. This has spurred interest in distilling knowledge from large VLMs into smaller, more efficient counterparts. A key challenge arises here from the diversity of VLM architectures, which are built on different LLMs and employ varying token types-differing in vocabulary size, token splits, and token index ordering. To address this challenge of limitation to a specific VLM type, we present Generation after Recalibration (GenRecal), a general-purpose distillation framework for VLMs. GenRecal incorporates a Recalibrator that aligns and adapts feature representations between heterogeneous VLMs, enabling effective knowledge transfer across different types of VLMs. Through extensive experiments on multiple challenging benchmarks, we demonstrate that GenRecal significantly improves baseline performances, eventually outperforming large-scale open- and closed-source VLMs.

Method: Trained over 100 LMs with 1B and 4B parameters from scratch, systematically analyzing training dynamics across pre-training, continued pre-training, supervised fine-tuning, and reinforcement learning phases.

Result: Key findings include diminishing returns from excessive pre-training/post-training, importance of mitigating forgetting during domain-specific continued pre-training, continued pre-training’s crucial role in bridging phases, and intricate trade-offs in fine-tuning and RL configuration.

Conclusion: EvoLM provides a comprehensive framework for transparent LM training analysis, with all models, datasets, and pipelines released to facilitate open research and reproducibility.

Abstract: Modern language model (LM) training has been divided into multiple stages, making it difficult for downstream developers to evaluate the impact of design choices made at each stage. We present EvoLM, a model suite that enables systematic and transparent analysis of LMs’ training dynamics across pre-training, continued pre-training, supervised fine-tuning, and reinforcement learning. We train over 100 LMs with 1B and 4B parameters from scratch, and evaluate both upstream (language modeling) and downstream (problem-solving) capabilities, including considerations of both in-domain and out-of-domain generalization. Key insights highlight the diminishing returns from excessive pre-training and post-training, the importance and practices of mitigating forgetting during domain-specific continued pre-training, the crucial role of continued pre-training in bridging pre-training and post-training phases, and various intricate trade-offs when configuring supervised fine-tuning and reinforcement learning. To facilitate open research and reproducibility, we release all pre-trained and post-trained models, training datasets for all stages, and our entire training and evaluation pipeline.

Method: Framed agent behavior steering as a model editing task called Behavior Editing. Created BehaviorBench, a multi-tier benchmark based on psychological moral theories to evaluate and edit behaviors across scenarios.

Result: Behavior Editing successfully steers agents toward target behaviors in specific scenarios and enables global moral alignment shifts. The approach works for both promoting ethical behavior and inducing harmful behavior, validated across various LLM models.

Conclusion: Behavior Editing offers a new paradigm for steering agent behavior with both promising applications and potential perils, requiring careful consideration of ethical implications.

Abstract: Agents based on Large Language Models (LLMs) have demonstrated strong capabilities across a wide range of tasks. However, deploying LLM-based agents in high-stakes domains comes with significant safety and ethical risks. Unethical behavior by these agents can directly result in serious real-world consequences, including physical harm and financial loss. To efficiently steer the ethical behavior of agents, we frame agent behavior steering as a model editing task, which we term Behavior Editing. Model editing is an emerging area of research that enables precise and efficient modifications to LLMs while preserving their overall capabilities. To systematically study and evaluate this approach, we introduce BehaviorBench, a multi-tier benchmark grounded in psychological moral theories. This benchmark supports both the evaluation and editing of agent behaviors across a variety of scenarios, with each tier introducing more complex and ambiguous scenarios. We first demonstrate that Behavior Editing can dynamically steer agents toward the target behavior within specific scenarios. Moreover, Behavior Editing enables not only scenario-specific local adjustments but also more extensive shifts in an agent’s global moral alignment. We demonstrate that Behavior Editing can be used to promote ethical and benevolent behavior or, conversely, to induce harmful or malicious behavior. Through extensive evaluations of agents built on frontier LLMs, BehaviorBench validates the effectiveness of behavior editing across a wide range of models and scenarios. Our findings offer key insights into a new paradigm for steering agent behavior, highlighting both the promise and perils of Behavior Editing.

Method: Proposed Difficulty-Influence Quadrant (DIQ) that prioritizes samples in the high-difficulty-high-influence quadrant to balance complex clinical reasoning with substantial gradient influence.

Result: DIQ enables models fine-tuned on only 1% of selected data to match full-dataset performance, while using 10% consistently outperforms baseline methods. Human and LLM evaluations show higher data quality and better alignment with expert clinical reasoning patterns.

Conclusion: DIQ demonstrates the superiority of principled data selection over brute-force scaling, enabling efficient medical reasoning with minimal fine-tuning data while maintaining expert-like reasoning patterns.

Abstract: Supervised Fine-Tuning (SFT) plays a pivotal role in adapting Large Language Models (LLMs) to specialized domains such as medical reasoning. However, existing SFT practices often rely on unfiltered datasets that contain redundant and low-quality samples, leading to substantial computational costs and suboptimal performance. Although existing methods attempt to alleviate this problem by selecting data based on sample difficulty, defined by knowledge and reasoning complexity, they overlook each sample’s optimization utility reflected in its gradient. Interestingly, we find that gradient-based influence alone favors easy-to-optimize samples that cause large parameter shifts but lack deep reasoning chains, while difficulty alone selects noisy or overly complex cases that fail to guide stable optimization. Based on this observation, we propose a data selection strategy, Difficulty-Influence Quadrant (DIQ), which prioritizes samples in the high-difficulty-high-influence quadrant to balance complex clinical reasoning with substantial gradient influence, enabling efficient medical reasoning with minimal fine-tuning data. Furthermore, Human and LLM-as-a-judge evaluations show that DIQ-selected subsets demonstrate higher data quality and generate clinical reasoning that is more aligned with expert practices in differential diagnosis, safety check, and evidence citation, as DIQ emphasizes samples that foster expert-like reasoning patterns. Extensive experiments on medical reasoning benchmarks demonstrate that DIQ enables models fine-tuned on only 1% of selected data to match full-dataset performance, while using 10% consistently outperforms baseline methods, highlighting the superiority of principled data selection over brute-force scaling. The code and data are available at https://github.com/mihara-bot/DIQ.

Method: Novel continuous sentiment scoring method based on concept vector projection, trained on multilingual literary data to capture nuanced sentiment expressions.

Result: Outperforms existing tools on English and Danish texts, producing sentiment scores whose distribution closely matches human ratings, enabling more accurate analysis and sentiment arc modeling in literature.

Conclusion: The proposed continuous sentiment scoring method effectively addresses the unique challenges of literary sentiment analysis and enables more accurate sentiment modeling across diverse literary contexts.

Abstract: Sentiment Analysis is widely used to quantify sentiment in text, but its application to literary texts poses unique challenges due to figurative language, stylistic ambiguity, as well as sentiment evocation strategies. Traditional dictionary-based tools often underperform, especially for low-resource languages, and transformer models, while promising, typically output coarse categorical labels that limit fine-grained analysis. We introduce a novel continuous sentiment scoring method based on concept vector projection, trained on multilingual literary data, which more effectively captures nuanced sentiment expressions across genres, languages, and historical periods. Our approach outperforms existing tools on English and Danish texts, producing sentiment scores whose distribution closely matches human ratings, enabling more accurate analysis and sentiment arc modeling in literature.

Method: Examined RALM calibration across different knowledge states, used in-context fine-tuning to mitigate over-refusal, and developed a refusal mechanism combining refusal-aware RALMs with uncertainty-based answer abstention.

Result: RALMs show over-refusal - refusing answerable questions when retrieved docs are irrelevant. In-context fine-tuning reduces over-refusal but doesn’t improve calibration. The refusal mechanism improves overall answer quality by balancing refusal and correct answers.

Conclusion: RALMs have poor calibration and over-refusal issues. Uncertainty estimation for RALMs remains an open problem requiring deeper investigation.

Abstract: Existing large language models (LLMs) occasionally generate plausible yet factually incorrect responses, known as hallucinations. Two main approaches have been proposed to mitigate hallucinations: retrieval-augmented language models (RALMs) and refusal post-training. However, current research predominantly focuses on their individual effectiveness while overlooking the evaluation of the refusal capability of RALMs. Ideally, if RALMs know when they do not know, they should refuse to answer.In this study, we ask the fundamental question: Do RALMs know when they don’t know? Specifically, we investigate three questions. First, are RALMs well calibrated with respect to different internal and external knowledge states? We examine the influence of various factors. Contrary to expectations, when all retrieved documents are irrelevant, RALMs still tend to refuse questions they could have answered correctly. Next, given the model’s pronounced \textbf{over-refusal} behavior, we raise a second question: How does a RALM’s refusal ability align with its calibration quality? Our results show that the over-refusal problem can be mitigated through in-context fine-tuning. However, we observe that improved refusal behavior does not necessarily imply better calibration or higher overall accuracy. Finally, we ask: Can we combine refusal-aware RALMs with uncertainty-based answer abstention to mitigate over-refusal? We develop a simple yet effective refusal mechanism for refusal-post-trained RALMs that improves their overall answer quality by balancing refusal and correct answers. Our study provides a more comprehensive understanding of the factors influencing RALM behavior. Meanwhile, we emphasize that uncertainty estimation for RALMs remains an open problem deserving deeper investigation.

Method: Pretrained 3 domain-specific masked language models using ModernBERT architecture on 60M+ patent records with optimizations like FlashAttention, rotary embeddings, and GLU layers.

Result: ModernBERT-base-PT outperforms general ModernBERT on 3/4 classification tasks, achieves competitive performance with PatentBERT, and all variants are 3x faster than PatentBERT.

Conclusion: Domain-specific pretraining with architectural improvements significantly enhances patent NLP performance while maintaining fast inference for time-sensitive applications.

Abstract: Transformer-based language models such as BERT have become foundational in NLP, yet their performance degrades in specialized domains like patents, which contain long, technical, and legally structured text. Prior approaches to patent NLP have primarily relied on fine-tuning general-purpose models or domain-adapted variants pretrained with limited data. In this work, we pretrain 3 domain-specific masked language models for patents, using the ModernBERT architecture and a curated corpus of over 60 million patent records. Our approach incorporates architectural optimizations, including FlashAttention, rotary embeddings, and GLU feed-forward layers. We evaluate our models on four downstream patent classification tasks. Our model, ModernBERT-base-PT, consistently outperforms the general-purpose ModernBERT baseline on three out of four datasets and achieves competitive performance with a baseline PatentBERT. Additional experiments with ModernBERT-base-VX and Mosaic-BERT-large demonstrate that scaling the model size and customizing the tokenizer further enhance performance on selected tasks. Notably, all ModernBERT variants retain substantially faster inference over - 3x that of PatentBERT - underscoring their suitability for time-sensitive applications. These results underscore the benefits of domain-specific pretraining and architectural improvements for patent-focused NLP tasks.

Method: The researchers created a bilingual English-Telugu dataset and experimented with different LLM-based approaches for veracity classification and justification generation.

Result: The paper benchmarks various veracity classification approaches using LLMs, though specific performance metrics are not provided in the abstract.

Conclusion: The work contributes a bilingual dataset and provides benchmarking of LLM approaches for automated fact-checking in multiple languages.

Abstract: False information poses a significant global challenge, and manually verifying claims is a time-consuming and resource-intensive process. In this research paper, we experiment with different approaches to investigate the effectiveness of large language models (LLMs) in classifying factual claims by their veracity and generating justifications in English and Telugu. The key contributions of this work include the creation of a bilingual English-Telugu dataset and the benchmarking of different veracity classification approaches based on LLMs.

Method: Combines chi-square statistical analysis for keyword extraction with adaptive majority voting for decision-making, exploring statistical vs random keyword selection and adaptive voting mechanisms.

Result: Achieves micro-F1 of 67.61, outperforming n-gram baselines (60.75) and random approaches (14.65). Chi-square keywords provide consistent improvements across categories.

Conclusion: Chi-square-based keywords show the most consistent impact across all categories, and adaptive voting benefits ambiguous cases requiring extended classification rounds.

Abstract: The BLP-2025 Task 1A requires Bengali hate speech classification into six categories. Traditional supervised approaches need extensive labeled datasets that are expensive for low-resource languages. We developed PromptGuard, a few-shot framework combining chi-square statistical analysis for keyword extraction with adaptive majority voting for decision-making. We explore statistical keyword selection versus random approaches and adaptive voting mechanisms that extend classification based on consensus quality. Chi-square keywords provide consistent improvements across categories, while adaptive voting benefits ambiguous cases requiring extended classification rounds. PromptGuard achieves a micro-F1 of 67.61, outperforming n-gram baselines (60.75) and random approaches (14.65). Ablation studies confirm chi-square-based keywords show the most consistent impact across all categories.

Method: Used Pangram AI detector to analyze 186K articles from 1.5K American newspapers and 45K opinion pieces from major publications, supplemented by manual audit of 100 AI-flagged articles.

Result: 9% of articles contain AI-generated content, with higher prevalence in local outlets, weather/technology topics, and opinion pieces (6.4x more likely than news articles). Only 5% of AI-flagged articles disclosed AI use.

Conclusion: Urgent need for greater transparency and updated editorial standards regarding AI use in journalism to maintain public trust.

Abstract: AI is rapidly transforming journalism, but the extent of its use in published newspaper articles remains unclear. We address this gap by auditing a large-scale dataset of 186K articles from online editions of 1.5K American newspapers published in the summer of 2025. Using Pangram, a state-of-the-art AI detector, we discover that approximately 9% of newly-published articles are either partially or fully AI-generated. This AI use is unevenly distributed, appearing more frequently in smaller, local outlets, in specific topics such as weather and technology, and within certain ownership groups. We also analyze 45K opinion pieces from Washington Post, New York Times, and Wall Street Journal, finding that they are 6.4 times more likely to contain AI-generated content than news articles from the same publications, with many AI-flagged op-eds authored by prominent public figures. Despite this prevalence, we find that AI use is rarely disclosed: a manual audit of 100 AI-flagged articles found only five disclosures of AI use. Overall, our audit highlights the immediate need for greater transparency and updated editorial standards regarding the use of AI in journalism to maintain public trust.

Method: Four-component system: SenseFusion (lightweight encoder for sensor-to-language alignment), Edge-RAG (local retrieval for site-specific grounding), PromptRouter (cost-aware policy for edge/cloud routing), and Secure Execution (data minimization with raw data staying on-device).

Result: Achieves sub-second (p95) latency on edge paths, reduces bandwidth costs through local retrieval, maintains privacy by transmitting only discrete codes, improves factual consistency, and lowers energy consumption through optimizations.

Conclusion: Edge-first design successfully treats semantics, privacy, and predictable latency as co-equal goals for large model deployments in challenging environments.

Abstract: We present CoSense-LLM, an edge-first framework that turns continuous multimodal sensor streams (for example Wi-Fi CSI, IMU, audio, RFID, and lightweight vision) into compact, verifiable semantic tokens and coordinates with large language models under explicit latency, energy, bandwidth, and privacy constraints. CoSense-LLM has four parts: (i) SenseFusion, a lightweight encoder that aligns sensor embeddings with language and compresses them into short discrete code sequences; (ii) Edge-RAG, a local hybrid retrieval layer that grounds generation in site specific policies and notes; (iii) PromptRouter, a cost and uncertainty aware policy that selects edge only generation, edge plus retrieval, or compact cloud escalation; and (iv) Secure Execution, an auditable redaction path that enforces data minimization so raw waveforms never leave the device. The system works with modern serving optimizations, including paged or streaming KV caches, FlashAttention style kernels, speculative decoding, and quantized LoRA adapters, and supports on device personalization and federated updates under non IID drift. Across home, office, and clinic deployments, CoSense-LLM delivers grounded explanations while meeting tight service level objectives: it sustains sub second (p95) end to end latency on edge dominant paths, reduces inter tier token and bandwidth costs by preferring local retrieval grounded responses, and preserves privacy by transmitting only discrete codes and redacted metadata. Ablations show that Edge-RAG improves factual consistency and reduces contradictions, calibrated uncertainty enables selective abstention and controlled escalations, and KV plus decoding accelerators lower energy per decision. The results support an edge first design that treats semantics, privacy, and predictable latency as co equal goals for large model deployments in interference prone environments.

Method: Three key components: (i) iterative computation in latent space, (ii) entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens of pre-training data.

Result: Ouro 1.4B and 2.6B models match the performance of up to 12B state-of-the-art LLMs across various benchmarks. The advantage comes from superior knowledge manipulation capabilities rather than increased knowledge capacity.

Conclusion: LoopLM represents a novel scaling direction for reasoning capabilities, with reasoning traces more aligned with final outputs than explicit chain-of-thought, showing potential for advancing reasoning in language models.

Abstract: Modern LLMs are trained to “think” primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training data. We present and open-source Ouro, named after the recursive Ouroboros, a family of pre-trained Looped Language Models (LoopLM) that instead build reasoning into the pre-training phase through (i) iterative computation in latent space, (ii) an entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and 2.6B models enjoy superior performance that match the results of up to 12B SOTA LLMs across a wide range of benchmarks. Through controlled experiments, we show this advantage stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities. We also show that LoopLM yields reasoning traces more aligned with final outputs than explicit CoT. We hope our results show the potential of LoopLM as a novel scaling direction in the reasoning era. Our model is available here: http://ouro-llm.github.io.

Method: Investigated BPE segmentation recipes across various data volumes and language pairs, comparing symmetric vs asymmetric BPE approaches with different numbers of merge operations.

Result: Asymmetric BPE significantly improved MT performance, with 5.32, 4.46, and 0.7 CHRF++ gains on English-Hindi in low-resource settings. Validated across 6 additional language pairs with significant improvements in 10/12 systems.

Conclusion: High NMO for source (4K-32K) and low NMO for target (0.5K-2K) provides optimal results, particularly benefiting low-resource machine translation.

Abstract: Existing Machine Translation (MT) research often suggests a single, fixed set of hyperparameters for word segmentation models, symmetric Byte Pair Encoding (BPE), which applies the same number of merge operations (NMO) to train tokenizers for both source and target languages. However, we demonstrate that this uniform approach doesn’t guarantee optimal MT performance across different language pairs and data sizes. This work investigates BPE segmentation recipes across various data volumes and language pairs to evaluate MT system performance. We find that utilizing asymmetric BPE, where the source and target languages have different NMOs, significantly improves results over the symmetric approach, especially in low-resource settings (50K, 100K, and 500K sentence pairs). Specifically, asymmetric BPE yield statistically significant ($p<0.05$) average gains of 5.32, 4.46, and 0.7 CHRF++ on English-Hindi in low-resource setups (50K, 100K, and 500K sentence pairs, respectively). We validated this trend across six additional language pairs (English and Telugu, Shona, Norwegian, Kyrgyz, Hausa, and Inuktitut), observing statistically significant improvement in 10 out of 12 systems compared to symmetric BPE. Our findings indicate a high NMO for the source (4K to 32K) and a low NMO for the target (0.5K to 2K) provides optimal results, particularly benefiting low-resource MT.

Method: Structures essays as argumentation graphs where claims are nodes, evidence is attached as properties, and edges encode support/attack relations. Uses LLMs for initial classification and scoring, then visualizes results with justifications and quantitative coherence measures.

Result: Developed a system that enables rapid exploration of argumentative quality while retaining human oversight, providing tools for better understanding of argumentative essays and their corresponding graphs in natural language.

Conclusion: IntelliProof successfully bridges the gap between structural semantics of argumentative essays and user understanding, offering an interactive approach to essay analysis that combines automated LLM processing with human oversight and transparent explanations.

Abstract: We present IntelliProof, an interactive system for analyzing argumentative essays through LLMs. IntelliProof structures an essay as an argumentation graph, where claims are represented as nodes, supporting evidence is attached as node properties, and edges encode supporting or attacking relations. Unlike existing automated essay scoring systems, IntelliProof emphasizes the user experience: each relation is initially classified and scored by an LLM, then visualized for enhanced understanding. The system provides justifications for classifications and produces quantitative measures for essay coherence. It enables rapid exploration of argumentative quality while retaining human oversight. In addition, IntelliProof provides a set of tools for a better understanding of an argumentative essay and its corresponding graph in natural language, bridging the gap between the structural semantics of argumentative essays and the user’s understanding of a given text.

Method: Used the ContArgA corpus annotations to perform zero-shot prompting experiments evaluating the importance of gold-annotated and predicted emotions and appraisals for subjective convincingness assessment.

Result: Categorical emotion information improves convincingness prediction, but the improvement is more pronounced with appraisals. Appraisals show greater advantage over basic emotion categories.

Conclusion: This is the first systematic comparison between emotion models for convincingness prediction, demonstrating the superiority of appraisals and providing insights for computational argumentation applications.

Abstract: The convincingness of an argument does not only depend on its structure (logos), the person who makes the argument (ethos), but also on the emotion that it causes in the recipient (pathos). While the overall intensity and categorical values of emotions in arguments have received considerable attention in the research community, we argue that the emotion an argument evokes in a recipient is subjective. It depends on the recipient’s goals, standards, prior knowledge, and stance. Appraisal theories lend themselves as a link between the subjective cognitive assessment of events and emotions. They have been used in event-centric emotion analysis, but their suitability for assessing argument convincingness remains unexplored. In this paper, we evaluate whether appraisal theories are suitable for emotion analysis in arguments by considering subjective cognitive evaluations of the importance and impact of an argument on its receiver. Based on the annotations in the recently published ContArgA corpus, we perform zero-shot prompting experiments to evaluate the importance of gold-annotated and predicted emotions and appraisals for the assessment of the subjective convincingness labels. We find that, while categorical emotion information does improve convincingness prediction, the improvement is more pronounced with appraisals. This work presents the first systematic comparison between emotion models for convincingness prediction, demonstrating the advantage of appraisals, providing insights for theoretical and practical applications in computational argumentation.

Method: LoopTool uses three modules: Greedy Capability Probing to diagnose model capabilities, Judgement-Guided Label Verification to correct annotation errors using an open-source judge model, and Error-Driven Data Expansion to generate challenging samples based on identified failures.

Result: An 8B model trained with LoopTool significantly outperforms its 32B data generator and achieves state-of-the-art results on BFCL-v3 and ACEBench benchmarks for its scale.

Conclusion: Closed-loop, self-refining data pipelines can dramatically enhance LLM tool-use capabilities, demonstrating the effectiveness of adaptive, model-aware data evolution frameworks.

Abstract: Augmenting Large Language Models (LLMs) with external tools enables them to execute complex, multi-step tasks. However, tool learning is hampered by the static synthetic data pipelines where data generation and model training are executed as two separate, non-interactive processes. This approach fails to adaptively focus on a model’s specific weaknesses and allows noisy labels to persist, degrading training efficiency. We introduce LoopTool, a fully automated, model-aware data evolution framework that closes this loop by tightly integrating data synthesis and model training. LoopTool iteratively refines both the data and the model through three synergistic modules: (1) Greedy Capability Probing (GCP) diagnoses the model’s mastered and failed capabilities; (2) Judgement-Guided Label Verification (JGLV) uses an open-source judge model to find and correct annotation errors, progressively purifying the dataset; and (3) Error-Driven Data Expansion (EDDE) generates new, challenging samples based on identified failures. This closed-loop process operates within a cost-effective, open-source ecosystem, eliminating dependence on expensive closed-source APIs. Experiments show that our 8B model trained with LoopTool significantly surpasses its 32B data generator and achieves new state-of-the-art results on the BFCL-v3 and ACEBench benchmarks for its scale. Our work demonstrates that closed-loop, self-refining data pipelines can dramatically enhance the tool-use capabilities of LLMs.

Method: Created LocalBench with 14,782 validated question-answer pairs across 526 U.S. counties in 49 states, integrating diverse sources including Census statistics, local subreddit discourse, and regional news, spanning physical, cognitive, and relational dimensions of locality. Evaluated 13 state-of-the-art LLMs under closed-book and web-augmented settings.

Result: Even best-performing models achieve only 56.8% accuracy on narrative-style questions and below 15.5% on numerical reasoning. Larger model size and web augmentation don’t guarantee better performance - search improves Gemini’s accuracy by +13.6% but reduces GPT-series performance by -11.4%.

Conclusion: There is an urgent need for language models that can support equitable, place-aware AI systems capable of engaging with the diverse, fine-grained realities of local communities across geographic and cultural contexts.

Abstract: Large language models (LLMs) have been widely evaluated on macro-scale geographic tasks, such as global factual recall, event summarization, and regional reasoning. Yet, their ability to handle hyper-local knowledge remains poorly understood. This gap is increasingly consequential as real-world applications, from civic platforms to community journalism, demand AI systems that can reason about neighborhood-specific dynamics, cultural narratives, and local governance. Existing benchmarks fall short in capturing this complexity, often relying on coarse-grained data or isolated references. We present LocalBench, the first benchmark designed to systematically evaluate LLMs on county-level local knowledge across the United States. Grounded in the Localness Conceptual Framework, LocalBench includes 14,782 validated question-answer pairs across 526 U.S. counties in 49 states, integrating diverse sources such as Census statistics, local subreddit discourse, and regional news. It spans physical, cognitive, and relational dimensions of locality. Using LocalBench, we evaluate 13 state-of-the-art LLMs under both closed-book and web-augmented settings. Our findings reveal critical limitations: even the best-performing models reach only 56.8% accuracy on narrative-style questions and perform below 15.5% on numerical reasoning. Moreover, larger model size and web augmentation do not guarantee better performance, for example, search improves Gemini’s accuracy by +13.6%, but reduces GPT-series performance by -11.4%. These results underscore the urgent need for language models that can support equitable, place-aware AI systems: capable of engaging with the diverse, fine-grained realities of local communities across geographic and cultural contexts.

Method: SpiderGen integrates traditional LCA taxonomy and methodology with LLM reasoning capabilities to generate graphical representations of LCA process information (PCR PFGs), evaluated against 65 real-world LCA documents.

Result: SpiderGen achieved 65% F1-Score across 10 sample data points, outperforming one-shot prompting (53%) and other baseline techniques. It produces accurate LCA process information with only minor errors.

Conclusion: SpiderGen demonstrates significant potential to reduce human effort and costs for carbon impact estimation, making LCA more accessible while maintaining reasonable accuracy compared to traditional methods.

Abstract: Investigating the effects of climate change and global warming caused by GHG emissions have been a key concern worldwide. These emissions are largely contributed to by the production, use and disposal of consumer products. Thus, it is important to build tools to estimate the environmental impact of consumer goods, an essential part of which is conducting Life Cycle Assessments (LCAs). LCAs specify and account for the appropriate processes involved with the production, use, and disposal of the products. We present SpiderGen, an LLM-based workflow which integrates the taxonomy and methodology of traditional LCA with the reasoning capabilities and world knowledge of LLMs to generate graphical representations of the key procedural information used for LCA, known as Product Category Rules Process Flow Graphs (PCR PFGs). We additionally evaluate the output of SpiderGen by comparing it with 65 real-world LCA documents. We find that SpiderGen provides accurate LCA process information that is either fully correct or has minor errors, achieving an F1-Score of 65% across 10 sample data points, as compared to 53% using a one-shot prompting method. We observe that the remaining errors occur primarily due to differences in detail between LCA documents, as well as differences in the “scope” of which auxiliary processes must also be included. We also demonstrate that SpiderGen performs better than several baselines techniques, such as chain-of-thought prompting and one-shot prompting. Finally, we highlight SpiderGen’s potential to reduce the human effort and costs for estimating carbon impact, as it is able to produce LCA process information for less than $1 USD in under 10 minutes as compared to the status quo LCA, which can cost over $25000 USD and take up to 21-person days.

Method: Used GPT-4o to evaluate short-answer quizzes and project reports from an undergraduate Computational Linguistics course, comparing LLM-generated scores against independent human evaluations by teaching assistants across 50 students and 14 teams.

Result: GPT-4o achieved strong correlation with human graders (up to 0.98) and exact score agreement in 55% of quiz cases. For project reports, it showed strong overall alignment with human grading but exhibited variability in scoring technical, open-ended responses.

Conclusion: LLM-based grading systems show significant potential for educational assessment but have limitations, particularly in handling technical open-ended responses. The work contributes to advancing automated grading in real-world academic settings.

Abstract: Large Language Models (LLMs) are increasingly explored for educational tasks such as grading, yet their alignment with human evaluation in real classrooms remains underexamined. In this study, we investigate the feasibility of using an LLM (GPT-4o) to evaluate short-answer quizzes and project reports in an undergraduate Computational Linguistics course. We collect responses from approximately 50 students across five quizzes and receive project reports from 14 teams. LLM-generated scores are compared against human evaluations conducted independently by the course teaching assistants (TAs). Our results show that GPT-4o achieves strong correlation with human graders (up to 0.98) and exact score agreement in 55% of quiz cases. For project reports, it also shows strong overall alignment with human grading, while exhibiting some variability in scoring technical, open-ended responses. We release all code and sample data to support further research on LLMs in educational assessment. This work highlights both the potential and limitations of LLM-based grading systems and contributes to advancing automated grading in real-world academic settings.

Method: Linking metadata from shadow libraries (Library Genesis, Z-Library) with structured bibliographic data from Goodreads, prioritizing natively digital EPUB files for machine-readability, and including secondary datasets for French, German, and Spanish.

Result: Created a high-precision corpus of over 539,000 English-language book references spanning three centuries, enriched with first publication dates, genres, ratings, and reviews, with evaluated linkage accuracy.

Conclusion: The project provides openly released data and discusses legal permissibility under EU and US frameworks for text and data mining in research, enabling broader access to cultural analytics resources.

Abstract: This data paper introduces MajinBook, an open catalogue designed to facilitate the use of shadow libraries–such as Library Genesis and Z-Library–for computational social science and cultural analytics. By linking metadata from these vast, crowd-sourced archives with structured bibliographic data from Goodreads, we create a high-precision corpus of over 539,000 references to English-language books spanning three centuries, enriched with first publication dates, genres, and popularity metrics like ratings and reviews. Our methodology prioritizes natively digital EPUB files to ensure machine-readable quality, while addressing biases in traditional corpora like HathiTrust, and includes secondary datasets for French, German, and Spanish. We evaluate the linkage strategy for accuracy, release all underlying data openly, and discuss the project’s legal permissibility under EU and US frameworks for text and data mining in research.

Method: Uses reinforcement learning to train models for efficient interaction scaling, allowing up to 600 tool calls per task within a 256K context window for sustained multi-turn reasoning.

Result: Achieves 81.9% on GAIA, 37.7% on HLE, 47.1% on BrowseComp, and 55.6% on BrowseComp-ZH, surpassing previous open-source agents and approaching commercial counterparts like GPT-5-high.

Conclusion: Interaction scaling demonstrates predictable performance improvements with deeper interactions, establishing it as a third critical dimension alongside model capacity and context windows for next-generation research agents.

Abstract: We present MiroThinker v1.0, an open-source research agent designed to advance tool-augmented reasoning and information-seeking capabilities. Unlike previous agents that only scale up model size or context length, MiroThinker explores interaction scaling at the model level, systematically training the model to handle deeper and more frequent agent-environment interactions as a third dimension of performance improvement. Unlike LLM test-time scaling, which operates in isolation and risks degradation with longer reasoning chains, interactive scaling leverages environment feedback and external information acquisition to correct errors and refine trajectories. Through reinforcement learning, the model achieves efficient interaction scaling: with a 256K context window, it can perform up to 600 tool calls per task, enabling sustained multi-turn reasoning and complex real-world research workflows. Across four representative benchmarks-GAIA, HLE, BrowseComp, and BrowseComp-ZH-the 72B variant achieves up to 81.9%, 37.7%, 47.1%, and 55.6% accuracy respectively, surpassing previous open-source agents and approaching commercial counterparts such as GPT-5-high. Our analysis reveals that MiroThinker benefits from interactive scaling consistently: research performance improves predictably as the model engages in deeper and more frequent agent-environment interactions, demonstrating that interaction depth exhibits scaling behaviors analogous to model size and context length. These findings establish interaction scaling as a third critical dimension for building next-generation open research agents, complementing model capacity and context windows.

Method: The paper systematically reviews MCoT from three perspectives: CoT paradigms, post-training stage, and inference stage, while analyzing their underlying mechanisms. It also summarizes evaluation benchmarks and metrics.

Result: The review provides comprehensive analysis of MCoT methods, their mechanisms, existing evaluation frameworks, and application scenarios in multimodal reasoning tasks.

Conclusion: The paper identifies current challenges facing MCoT and provides an outlook on future research directions for advancing multimodal reasoning capabilities through chain-of-thought approaches.

Abstract: With the remarkable success of Multimodal Large Language Models (MLLMs) in perception tasks, enhancing their complex reasoning capabilities has emerged as a critical research focus. Existing models still suffer from challenges such as opaque reasoning paths and insufficient generalization ability. Chain-of-Thought (CoT) reasoning, which has demonstrated significant efficacy in language models by enhancing reasoning transparency and output interpretability, holds promise for improving model reasoning capabilities when extended to the multimodal domain. This paper provides a systematic review centered on “Multimodal Chain-of-Thought” (MCoT). First, it analyzes the background and theoretical motivations for its inception from the perspectives of technical evolution and task demands. Then, it introduces mainstream MCoT methods from three aspects: CoT paradigms, the post-training stage, and the inference stage, while also analyzing their underlying mechanisms. Furthermore, the paper summarizes existing evaluation benchmarks and metrics, and discusses the application scenarios of MCoT. Finally, it analyzes the challenges currently facing MCoT and provides an outlook on its future research directions.

Method: Three-stage training: 1) Continuous pre-training on mathematical corpus with novel data tasks including CoT-augmented state prediction, 2) Supervised Fine-tuning with expert iteration loop, 3) Group Relative Policy Optimization for challenging problems.

Result: State-of-the-art performance among similarly-sized open-source models, solving 27 problems on PutnamBench (pass@32) and achieving 24.0% on CombiBench (pass@32).

Conclusion: The diverse training data and progressively refined training pipeline effectively enhances formal reasoning capabilities of lightweight LLMs.

Abstract: Large Language Models (LLMs) have shown significant promise in automated theorem proving, yet progress is often constrained by the scarcity of diverse and high-quality formal language data. To address this issue, we introduce Spark-Prover-X1, a 7B parameter model trained via an three-stage framework designed to unlock the reasoning potential of more accessible and moderately-sized LLMs. The first stage infuses deep knowledge through continuous pre-training on a broad mathematical corpus, enhanced by a suite of novel data tasks. Key innovation is a “CoT-augmented state prediction” task to achieve fine-grained reasoning. The second stage employs Supervised Fine-tuning (SFT) within an expert iteration loop to specialize both the Spark-Prover-X1-7B and Spark-Formalizer-X1-7B models. Finally, a targeted round of Group Relative Policy Optimization (GRPO) is applied to sharpen the prover’s capabilities on the most challenging problems. To facilitate robust evaluation, particularly on problems from real-world examinations, we also introduce ExamFormal-Bench, a new benchmark dataset of 402 formal problems. Experimental results demonstrate that Spark-Prover achieves state-of-the-art performance among similarly-sized open-source models within the “Whole-Proof Generation” paradigm. It shows exceptional performance on difficult competition benchmarks, notably solving 27 problems on PutnamBench (pass@32) and achieving 24.0% on CombiBench (pass@32). Our work validates that this diverse training data and progressively refined training pipeline provides an effective path for enhancing the formal reasoning capabilities of lightweight LLMs. Both Spark-Prover-X1-7B and Spark-Formalizer-X1-7B, along with the ExamFormal-Bench dataset, are made publicly available at: https://www.modelscope.cn/organization/iflytek, https://gitcode.com/ifly_opensource.

Method: Tested multiple LLMs (GPT-3.5, GPT-4, GPT-4o, Gemini 1.0 Pro, Llama 2/3, Mixtral 8x7B, airoboros 70B, RoLlama 2 7B) using comprehensive corpus with various prompt templates from zero-shot to multi-shot instructions.

Result: GPT-4o achieves high diacritic restoration accuracy, consistently surpassing neutral echo baseline, while Meta’s Llama family exhibits wider variability.

Conclusion: Model architecture, training data, and prompt design significantly impact diacritic restoration performance, outlining directions for improving NLP tools for diacritic-rich languages.

Abstract: Automatic diacritic restoration is crucial for text processing in languages with rich diacritical marks, such as Romanian. This study evaluates the performance of several large language models (LLMs) in restoring diacritics in Romanian texts. Using a comprehensive corpus, we tested models including OpenAI’s GPT-3.5, GPT-4, GPT-4o, Google’s Gemini 1.0 Pro, Meta’s Llama 2 and Llama 3, MistralAI’s Mixtral 8x7B Instruct, airoboros 70B, and OpenLLM-Ro’s RoLlama 2 7B, under multiple prompt templates ranging from zero-shot to complex multi-shot instructions. Results show that models such as GPT-4o achieve high diacritic restoration accuracy, consistently surpassing a neutral echo baseline, while others, including Meta’s Llama family, exhibit wider variability. These findings highlight the impact of model architecture, training data, and prompt design on diacritic restoration performance and outline promising directions for improving NLP tools for diacritic-rich languages.

Method: O-Mem is based on active user profiling that dynamically extracts and updates user characteristics and event records from proactive interactions. It supports hierarchical retrieval of persona attributes and topic-related context.

Result: O-Mem achieves 51.67% on LoCoMo benchmark (3% improvement over LangMem) and 62.99% on PERSONAMEM (3.5% improvement over A-Mem). It also boosts token and interaction response time efficiency compared to previous frameworks.

Conclusion: O-Mem opens up promising directions for developing efficient and human-like personalized AI assistants by addressing key challenges in long-term interaction consistency and personalization.

Abstract: Recent advancements in LLM-powered agents have demonstrated significant potential in generating human-like responses; however, they continue to face challenges in maintaining long-term interactions within complex environments, primarily due to limitations in contextual consistency and dynamic personalization. Existing memory systems often depend on semantic grouping prior to retrieval, which can overlook semantically irrelevant yet critical user information and introduce retrieval noise. In this report, we propose the initial design of O-Mem, a novel memory framework based on active user profiling that dynamically extracts and updates user characteristics and event records from their proactive interactions with agents. O-Mem supports hierarchical retrieval of persona attributes and topic-related context, enabling more adaptive and coherent personalized responses. O-Mem achieves 51.67% on the public LoCoMo benchmark, a nearly 3% improvement upon LangMem,the previous state-of-the-art, and it achieves 62.99% on PERSONAMEM, a 3.5% improvement upon A-Mem,the previous state-of-the-art. O-Mem also boosts token and interaction response time efficiency compared to previous memory frameworks. Our work opens up promising directions for developing efficient and human-like personalized AI assistants in the future.

Method: Translation and Image Generation (TAI) framework with: (1) translation module using Odds Ratio Preference Alignment Algorithm, (2) image generation module using semantic graphs to capture metaphors and meanings, and creation of MorphoVerse dataset with 1,570 poems across 21 Indian languages.

Result: TAI Diffusion outperforms strong baselines in poem image generation tasks based on comprehensive human and quantitative evaluations.

Conclusion: The work addresses poetry translation and visual comprehension gaps, broadening accessibility and enriching reader experience for culturally rich Indian-language poetry.

Abstract: Indian poetry, known for its linguistic complexity and deep cultural resonance, has a rich and varied heritage spanning thousands of years. However, its layered meanings, cultural allusions, and sophisticated grammatical constructions often pose challenges for comprehension, especially for non-native speakers or readers unfamiliar with its context and language. Despite its cultural significance, existing works on poetry have largely overlooked Indian language poems. In this paper, we propose the Translation and Image Generation (TAI) framework, leveraging Large Language Models (LLMs) and Latent Diffusion Models through appropriate prompt tuning. Our framework supports the United Nations Sustainable Development Goals of Quality Education (SDG 4) and Reduced Inequalities (SDG 10) by enhancing the accessibility of culturally rich Indian-language poetry to a global audience. It includes (1) a translation module that uses an Odds Ratio Preference Alignment Algorithm to accurately translate morphologically rich poetry into English, and (2) an image generation module that employs a semantic graph to capture tokens, dependencies, and semantic relationships between metaphors and their meanings, to create visually meaningful representations of Indian poems. Our comprehensive experimental evaluation, including both human and quantitative assessments, demonstrates the superiority of TAI Diffusion in poem image generation tasks, outperforming strong baselines. To further address the scarcity of resources for Indian-language poetry, we introduce the Morphologically Rich Indian Language Poems MorphoVerse Dataset, comprising 1,570 poems across 21 low-resource Indian languages. By addressing the gap in poetry translation and visual comprehension, this work aims to broaden accessibility and enrich the reader’s experience.

Method: Created nuCarla - a nuScenes-style BEV perception dataset within CARLA simulator with full nuScenes format compatibility, comparable scale to nuScenes but with more balanced class distributions, and direct usability for closed-loop simulation deployment.

Result: Provides high-performance BEV backbones achieving state-of-the-art detection results, and offers both data and models as open benchmarks that substantially accelerate closed-loop E2E development.

Conclusion: nuCarla paves the way toward reliable and safety-aware research in autonomous driving by addressing the dataset gap for closed-loop E2E autonomous driving simulation.

Abstract: End-to-end (E2E) autonomous driving heavily relies on closed-loop simulation, where perception, planning, and control are jointly trained and evaluated in interactive environments. Yet, most existing datasets are collected from the real world under non-interactive conditions, primarily supporting open-loop learning while offering limited value for closed-loop testing. Due to the lack of standardized, large-scale, and thoroughly verified datasets to facilitate learning of meaningful intermediate representations, such as bird’s-eye-view (BEV) features, closed-loop E2E models remain far behind even simple rule-based baselines. To address this challenge, we introduce nuCarla, a large-scale, nuScenes-style BEV perception dataset built within the CARLA simulator. nuCarla features (1) full compatibility with the nuScenes format, enabling seamless transfer of real-world perception models; (2) a dataset scale comparable to nuScenes, but with more balanced class distributions; (3) direct usability for closed-loop simulation deployment; and (4) high-performance BEV backbones that achieve state-of-the-art detection results. By providing both data and models as open benchmarks, nuCarla substantially accelerates closed-loop E2E development, paving the way toward reliable and safety-aware research in autonomous driving.

Method: Two-component framework: 1) perturbation-based uncertainty estimation with controllable parameter perturbations, 2) two-stage unknown detection with learning-based classifiers leveraging estimated uncertainty.

Result: Superior performance over existing OSR methods demonstrated on three public datasets.

Conclusion: The proposed framework effectively mitigates overconfidence caused by inter-class overlap and enhances Open Set Recognition performance.

Abstract: Open Set Recognition (OSR) requires models not only to accurately classify known classes but also to effectively reject unknown samples. However, when unknown samples are semantically similar to known classes, inter-class overlap in the feature space often causes models to assign unjustifiably high confidence to them, leading to misclassification as known classes – a phenomenon known as overconfidence. This overconfidence undermines OSR by blurring the decision boundary between known and unknown classes. To address this issue, we propose a framework that explicitly mitigates overconfidence caused by inter-class overlap. The framework consists of two components: a perturbation-based uncertainty estimation module, which applies controllable parameter perturbations to generate diverse predictions and quantify predictive uncertainty, and an unknown detection module with distinct learning-based classifiers, implemented as a two-stage procedure, which leverages the estimated uncertainty to improve discrimination between known and unknown classes, thereby enhancing OSR performance. Experimental results on three public datasets show that the proposed framework achieves superior performance over existing OSR methods.

Method: Decompose video into keyframe, motion, and residual components; use dedicated encoders for each component with a shared 3D decoder; employ decoupled adaptation strategy that freezes partial encoders while training others sequentially.

Result: Extensive experiments demonstrate superior video reconstruction performance compared to existing methods.

Conclusion: DeCo-VAE effectively achieves compact latent representation through explicit decoupling of video components and dedicated encoding, leading to improved video reconstruction quality.

Abstract: Existing video Variational Autoencoders (VAEs) generally overlook the similarity between frame contents, leading to redundant latent modeling. In this paper, we propose decoupled VAE (DeCo-VAE) to achieve compact latent representation. Instead of encoding RGB pixels directly, we decompose video content into distinct components via explicit decoupling: keyframe, motion and residual, and learn dedicated latent representation for each. To avoid cross-component interference, we design dedicated encoders for each decoupled component and adopt a shared 3D decoder to maintain spatiotemporal consistency during reconstruction. We further utilize a decoupled adaptation strategy that freezes partial encoders while training the others sequentially, ensuring stable training and accurate learning of both static and dynamic features. Extensive quantitative and qualitative experiments demonstrate that DeCo-VAE achieves superior video reconstruction performance.

Method: Uses object-aware temporal modeling with a filtering mechanism to selectively propagate high-confidence object features across frames. Combines few-shot trained detection and classification heads with focused feature propagation, eliminating the need for explicit object tube proposals.

Result: Achieves significant AP improvements: 3.7% (FSVOD-500), 5.3% (FSYTV-40), 4.3% (VidOR), and 4.5% (VidVRD) in 5-shot setting. Also shows improvements in 1-shot, 3-shot, and 10-shot configurations.

Conclusion: The proposed approach effectively addresses temporal consistency challenges in few-shot video object detection while achieving performance gains across multiple datasets and shot configurations without relying on complex region proposals.

Abstract: Few-shot Video Object Detection (FSVOD) addresses the challenge of detecting novel objects in videos with limited labeled examples, overcoming the constraints of traditional detection methods that require extensive training data. This task presents key challenges, including maintaining temporal consistency across frames affected by occlusion and appearance variations, and achieving novel object generalization without relying on complex region proposals, which are often computationally expensive and require task-specific training. Our novel object-aware temporal modeling approach addresses these challenges by incorporating a filtering mechanism that selectively propagates high-confidence object features across frames. This enables efficient feature progression, reduces noise accumulation, and enhances detection accuracy in a few-shot setting. By utilizing few-shot trained detection and classification heads with focused feature propagation, we achieve robust temporal consistency without depending on explicit object tube proposals. Our approach achieves performance gains, with AP improvements of 3.7% (FSVOD-500), 5.3% (FSYTV-40), 4.3% (VidOR), and 4.5 (VidVRD) in the 5-shot setting. Further results demonstrate improvements in 1-shot, 3-shot, and 10-shot configurations. We make the code public at: https://github.com/yogesh-iitj/fs-video-vit

Method: Integrate text descriptions into training and inference of diffusion-based depth estimation models (Marigold, Lotus, E2E-FT). Train on HyperSim and Virtual KITTI, evaluate on multiple datasets including NYUv2, KITTI, ETH3D, ScanNet, and DIODE.

Result: Strategy improves overall monocular depth estimation accuracy, especially in small areas. Enhances depth perception of specific regions described in text. Depth prediction can be iteratively refined with more detailed text. Language constraints accelerate convergence of both training and inference diffusion trajectory.

Conclusion: Language can effectively enhance monocular depth estimation by providing additional constraints that reduce ambiguity and improve accuracy, with the ability to iteratively refine predictions through more detailed text descriptions.

Abstract: Traditional monocular depth estimation suffers from inherent ambiguity and visual nuisances. We demonstrate that language can enhance monocular depth estimation by providing an additional condition (rather than images alone) aligned with plausible 3D scenes, thereby reducing the solution space for depth estimation. This conditional distribution is learned during the text-to-image pre-training of diffusion models. To generate images under various viewpoints and layouts that precisely reflect textual descriptions, the model implicitly models object sizes, shapes, and scales, their spatial relationships, and the overall scene structure. In this paper, Iris, we investigate the benefits of our strategy to integrate text descriptions into training and inference of diffusion-based depth estimation models. We experiment with three different diffusion-based monocular depth estimators (Marigold, Lotus, and E2E-FT) and their variants. By training on HyperSim and Virtual KITTI, and evaluating on NYUv2, KITTI, ETH3D, ScanNet, and DIODE, we find that our strategy improves the overall monocular depth estimation accuracy, especially in small areas. It also improves the model’s depth perception of specific regions described in the text. We find that by providing more details in the text, the depth prediction can be iteratively refined. Simultaneously, we find that language can act as a constraint to accelerate the convergence of both training and the inference diffusion trajectory. Code and generated text data will be released upon acceptance.

Method: Weakly-supervised audio-conditioned segmentation using foundation models (CLIP and SAM2) with egocentric cues (objects in hands) to identify potential collision sound sources.

Result: Outperforms competitive baselines by 3× and 4.7× in mIoU on two new benchmarks: EPIC-CS3 and Ego4D-CS3.

Conclusion: The proposed CS3 task and method effectively segment collision sound sources in egocentric videos using multisensory perception and foundation models.

Abstract: Humans excel at multisensory perception and can often recognise object properties from the sound of their interactions. Inspired by this, we propose the novel task of Collision Sound Source Segmentation (CS3), where we aim to segment the objects responsible for a collision sound in visual input (i.e. video frames from the collision clip), conditioned on the audio. This task presents unique challenges. Unlike isolated sound events, a collision sound arises from interactions between two objects, and the acoustic signature of the collision depends on both. We focus on egocentric video, where sounds are often clear, but the visual scene is cluttered, objects are small, and interactions are brief. To address these challenges, we propose a weakly-supervised method for audio-conditioned segmentation, utilising foundation models (CLIP and SAM2). We also incorporate egocentric cues, i.e. objects in hands, to find acting objects that can potentially be collision sound sources. Our approach outperforms competitive baselines by $3\times$ and $4.7\times$ in mIoU on two benchmarks we introduce for the CS3 task: EPIC-CS3 and Ego4D-CS3.

Method: Formulates fusion as probabilistic transport using flow matching, collects pseudo-labels from SOTA models with task-aware selection, uses Fusion Refiner to enhance degraded components, and employs elastic weight consolidation for multi-task learning.

Result: Achieves competitive performance across diverse fusion tasks with significantly improved sampling efficiency and maintains lightweight model design.

Conclusion: The proposed method provides an efficient and scalable solution for multi-modal image fusion with improved sampling efficiency and structural consistency.

Abstract: Current multi-modal image fusion methods typically rely on task-specific models, leading to high training costs and limited scalability. While generative methods provide a unified modeling perspective, they often suffer from slow inference due to the complex sampling trajectories from noise to image. To address this, we formulate image fusion as a direct probabilistic transport from source modalities to the fused image distribution, leveraging the flow matching paradigm to improve sampling efficiency and structural consistency. To mitigate the lack of high-quality fused images for supervision, we collect fusion results from multiple state-of-the-art models as priors, and employ a task-aware selection function to select the most reliable pseudo-labels for each task. We further introduce a Fusion Refiner module that employs a divide-and-conquer strategy to systematically identify, decompose, and enhance degraded components in selected pseudo-labels. For multi-task scenarios, we integrate elastic weight consolidation and experience replay mechanisms to preserve cross-task performance and enhance continual learning ability from both parameter stability and memory retention perspectives. Our approach achieves competitive performance across diverse fusion tasks, while significantly improving sampling efficiency and maintaining a lightweight model design. The code will be available at: https://github.com/Ist-Zhy/FusionFM.

Method: Two-stage framework: 1) Mapfusion diffusion model adds noise to road segment maps then denoises with temporal embeddings and ControlNet for context, 2) Crash detection by comparing monitored maps with generated maps.

Result: Mapfusion successfully generates realistic road segment evolution maps based on learned motion patterns, robust across sampling intervals, and effectively detects real-world crashes.

Conclusion: The proposed two-stage trajectory-free method is effective for accurate crash detection without requiring vehicle trajectory data.

Abstract: Real-time crash detection is essential for developing proactive safety management strategy and enhancing overall traffic efficiency. To address the limitations associated with trajectory acquisition and vehicle tracking, road segment maps recording the individual-level traffic dynamic data were directly served in crash detection. A novel two-stage trajectory-free crash detection framework, was present to generate the rational future road segment map and identify crashes. The first-stage diffusion-based segment map generation model, Mapfusion, conducts a noisy-to-normal process that progressively adds noise to the road segment map until the map is corrupted to pure Gaussian noise. The denoising process is guided by sequential embedding components capturing the temporal dynamics of segment map sequences. Furthermore, the generation model is designed to incorporate background context through ControlNet to enhance generation control. Crash detection is achieved by comparing the monitored segment map with the generations from diffusion model in second stage. Trained on non-crash vehicle motion data, Mapfusion successfully generates realistic road segment evolution maps based on learned motion patterns and remains robust across different sampling intervals. Experiments on real-world crashes indicate the effectiveness of the proposed two-stage method in accurately detecting crashes.

Method: Uses spectrum decomposition to separate frequency components, hierarchical Hilbert encoding, and adaptive training strategy that starts with coarse information then refines to fine features.

Result: Extensive experiments demonstrate effectiveness and efficiency of the proposed training methods for seismic foundation models.

Conclusion: Highlights importance of data encoding and training strategies tailored to seismic data’s distinct frequency characteristics for enhancing visual seismic foundation model pretraining.

Abstract: Due to the emergency and homogenization of Artificial Intelligence (AI) technology development, transformer-based foundation models have revolutionized scientific applications, such as drug discovery, materials research, and astronomy. However, seismic data presents unique characteristics that require specialized processing techniques for pretraining foundation models in seismic contexts with high- and low-frequency features playing crucial roles. Existing vision transformers (ViTs) with sequential tokenization ignore the intrinsic pattern and fail to grasp both the high- and low-frequency seismic information efficiently and effectively. This work introduces a novel adaptive two-grid foundation model training strategy (ADATG) with Hilbert encoding specifically tailored for seismogram data, leveraging the hierarchical structures inherent in seismic data. Specifically, our approach employs spectrum decomposition to separate high- and low-frequency components and utilizes hierarchical Hilbert encoding to represent the data effectively. Moreover, observing the frequency principle observed in ViTs, we propose an adaptive training strategy that initially emphasizes coarse-level information and then progressively refines the model’s focus on fine-level features. Our extensive experiments demonstrate the effectiveness and efficiency of our training methods. This research highlights the importance of data encoding and training strategies informed by the distinct characteristics of high- and low-frequency features in seismic images, ultimately contributing to the enhancement of visual seismic foundation models pretraining.

Method: Analyze temporal facial kinematics (blink dynamics, mouth/jaw motions, gaze variability, head adjustments) by stabilizing signals, converting to microdynamic time series, smoothing, and summarizing into clip-level statistics that capture motion distribution across streams.

Result: Achieved AUROC 0.953, AP 0.961, F1-score 0.851, accuracy 0.857 on YT DemTalk dataset; identified gaze lability and mouth/jaw dynamics as most informative cues.

Conclusion: Facial micro-dynamics analysis provides effective language-free dementia screening from in-the-wild videos, enabling scalable passive monitoring without clinical intervention.

Abstract: We target passive dementia screening from short camera-facing talking head video, developing a facial temporal micro dynamics analysis for language free detection of early neuro cognitive change. This enables unscripted, in the wild video analysis at scale to capture natural facial behaviors, transferrable across devices, topics, and cultures without active intervention by clinicians or researchers during recording. Most existing resources prioritize speech or scripted interviews, limiting use outside clinics and coupling predictions to language and transcription. In contrast, we identify and analyze whether temporal facial kinematics, including blink dynamics, small mouth jaw motions, gaze variability, and subtle head adjustments, are sufficient for dementia screening without speech or text. By stabilizing facial signals, we convert these micro movements into interpretable facial microdynamic time series, smooth them, and summarize short windows into compact clip level statistics for screening. Each window is encoded by its activity mix (the relative share of motion across streams), thus the predictor analyzes the distribution of motion across streams rather than its magnitude, making per channel effects transparent. We also introduce YT DemTalk, a new dataset curated from publicly available, in the wild camera facing videos. It contains 300 clips (150 with self reported dementia, 150 controls) to test our model and offer a first benchmarking of the corpus. On YT DemTalk, ablations identify gaze lability and mouth/jaw dynamics as the most informative cues, and light weighted shallow classifiers could attain a dementia prediction performance of (AUROC) 0.953, 0.961 Average Precision (AP), 0.851 F1-score, and 0.857 accuracy.

Method: Developed Gen-ViRe framework grounded in cognitive science, decomposing CoF reasoning into 6 cognitive dimensions and 24 subtasks, using multi-source data curation, minimal prompting protocols, and hybrid VLM-assisted evaluation with detailed criteria.

Result: Experiments on SOTA systems revealed substantial discrepancies between impressive visual quality and actual reasoning depth, establishing baselines and diagnostic tools.

Conclusion: Gen-ViRe provides the first quantitative assessment of video models as reasoners, enabling principled guidance for advancing genuine world simulators beyond visual quality to actual reasoning capabilities.

Abstract: While Chain-of-Thought (CoT) prompting enables sophisticated symbolic reasoning in LLMs, it remains confined to discrete text and cannot simulate the continuous, physics-governed dynamics of the real world. Recent video generation models have emerged as potential world simulators through Chain-of-Frames (CoF) reasoning – materializing thought as frame-by-frame visual sequences, with each frame representing a physically-grounded reasoning step. Despite compelling demonstrations, a challenge persists: existing benchmarks, focusing on fidelity or alignment, do not assess CoF reasoning and thus cannot measure core cognitive abilities in multi-step planning, algorithmic logic, or abstract pattern extrapolation. This evaluation void prevents systematic understanding of model capabilities and principled guidance for improvement. We introduce Gen-ViRe (Generative Visual Reasoning Benchmark), a framework grounded in cognitive science and real-world AI applications, which decomposes CoF reasoning into six cognitive dimensions – from perceptual logic to abstract planning – and 24 subtasks. Through multi-source data curation, minimal prompting protocols, and hybrid VLM-assisted evaluation with detailed criteria, Gen-ViRe delivers the first quantitative assessment of video models as reasoners. Our experiments on SOTA systems reveal substantial discrepancies between impressive visual quality and actual reasoning depth, establishing baselines and diagnostic tools to advance genuine world simulators.

Method: Propose ranking-based losses that increase correlation between confidence scores and localization quality, aligning with mAP evaluation metric. Applied to both 1D and 2D heatmaps across multiple datasets.

Result: RSPose achieves 79.9 mAP with ViTPose-H on COCO-val, outperforming previous SOTA. Also improves SimCC Resnet-50 by 1.5 AP to 73.6 AP on COCO-val. Shows superior performance to MSE and KL-Divergence losses.

Conclusion: Ranking-based losses effectively address key issues in heatmap-based pose estimation, achieving better correlation between confidence and localization, leading to improved mAP performance across multiple models and datasets.

Abstract: While heatmap-based human pose estimation methods have shown strong performance, they suffer from three main problems: (P1) “Commonly used Mean Squared Error (MSE)” Loss may not always improve joint localization because it penalizes all pixel deviations equally, without focusing explicitly on sharpening and correctly localizing the peak corresponding to the joint; (P2) heatmaps are spatially and class-wise imbalanced; and, (P3) there is a discrepancy between the evaluation metric (i.e., mAP) and the loss functions. We propose ranking-based losses to address these issues. Both theoretically and empirically, we show that our proposed losses are superior to commonly used heatmap losses (MSE, KL-Divergence). Our losses considerably increase the correlation between confidence scores and localization qualities, which is desirable because higher correlation leads to more accurate instance selection during Non-Maximum Suppression (NMS) and better Average Precision (mAP) performance. We refer to the models trained with our losses as RSPose. We show the effectiveness of RSPose across two different modes: one-dimensional and two-dimensional heatmaps, on three different datasets (COCO, CrowdPose, MPII). To the best of our knowledge, we are the first to propose losses that align with the evaluation metric (mAP) for human pose estimation. RSPose outperforms the previous state of the art on the COCO-val set and achieves an mAP score of 79.9 with ViTPose-H, a vision transformer model for human pose estimation. We also improve SimCC Resnet-50, a coordinate classification-based pose estimation method, by 1.5 AP on the COCO-val set, achieving 73.6 AP.

Method: Reformulates APR as inverse novel view synthesis, predicting two disentangled geometric representations: ray bundle directions for rotation and pointmaps for translation, then using a differentiable solver to recover the final 6-DoF pose.

Result: Achieves state-of-the-art performance on 7-Scenes and Cambridge Landmarks datasets, with explicit decoupling of rotation and translation predictions boosting performance.

Conclusion: Modeling the inverse rendering process through geometric representation regression provides a more robust path toward generalizable absolute pose estimation than black-box APR approaches.

Abstract: Absolute Pose Regression (APR) has emerged as a compelling paradigm for visual localization. However, APR models typically operate as black boxes, directly regressing a 6-DoF pose from a query image, which can lead to memorizing training views rather than understanding 3D scene geometry. In this work, we propose a geometrically-grounded alternative. Inspired by novel view synthesis, which renders images from intermediate geometric representations, we reformulate APR as its inverse that regresses the underlying 3D representations directly from the image, and we name this paradigm Geometric Representation Regression (GRR). Our model explicitly predicts two disentangled geometric representations in the world coordinate system: (1) a ray bundle’s directions to estimate camera rotation, and (2) a corresponding pointmap to estimate camera translation. The final 6-DoF camera pose is then recovered from these geometric components using a differentiable deterministic solver. This disentangled approach, which separates the learned visual-to-geometry mapping from the final pose calculation, introduces a strong geometric prior into the network. We find that the explicit decoupling of rotation and translation predictions measurably boosts performance. We demonstrate state-of-the-art performance on 7-Scenes and Cambridge Landmarks datasets, validating that modeling the inverse rendering process is a more robust path toward generalizable absolute pose estimation.

Method: Proposed H-CNN-ViT model with hierarchical gated attention that selectively weights features from global (ViT) and local (CNN) paths, processing each MRI modality independently for optimal feature integration.

Result: Achieved 78.6% AUC on their curated dataset, surpassing state-of-the-art models for bladder cancer recurrence prediction.

Conclusion: The introduced dataset and H-CNN-ViT model provide valuable benchmarks for bladder cancer recurrence research, with the model demonstrating superior performance through balanced feature fusion.

Abstract: Bladder cancer is one of the most prevalent malignancies worldwide, with a recurrence rate of up to 78%, necessitating accurate post-operative monitoring for effective patient management. Multi-sequence contrast-enhanced MRI is commonly used for recurrence detection; however, interpreting these scans remains challenging, even for experienced radiologists, due to post-surgical alterations such as scarring, swelling, and tissue remodeling. AI-assisted diagnostic tools have shown promise in improving bladder cancer recurrence prediction, yet progress in this field is hindered by the lack of dedicated multi-sequence MRI datasets for recurrence assessment study. In this work, we first introduce a curated multi-sequence, multi-modal MRI dataset specifically designed for bladder cancer recurrence prediction, establishing a valuable benchmark for future research. We then propose H-CNN-ViT, a new Hierarchical Gated Attention Multi-Branch model that enables selective weighting of features from the global (ViT) and local (CNN) paths based on contextual demands, achieving a balanced and targeted feature fusion. Our multi-branch architecture processes each modality independently, ensuring that the unique properties of each imaging channel are optimally captured and integrated. Evaluated on our dataset, H-CNN-ViT achieves an AUC of 78.6%, surpassing state-of-the-art models. Our model is publicly available at https://github.com/XLIAaron/H-CNN-ViT}.

Method: Replace CLIP’s text encoder with LLM-based embedding module (Qwen3-Embedding), introduce learnable prompts to enhance cross-modal alignment, leveraging LLMs’ extended context window and richer representations.

Result: Captures comprehensive medical semantics from long-form clinical text, substantially improves medical image-text alignment and downstream performance on radiology benchmarks.

Conclusion: QwenCLIP framework successfully addresses limitations of existing methods by leveraging LLMs’ capabilities for better handling of long radiology reports and improving medical vision-language tasks.

Abstract: Contrastive Language-Image Pretraining (CLIP) has demonstrated strong generalization for vision-language tasks in computer vision and medical domains, yet its text encoder accepts only up to 77 tokens, which limits its ability to represent long and information-rich radiology reports. Recent adaptations using domain-specific encoders, such as PubMedBERT or ClinicalBERT, mitigate this issue by leveraging medical corpora, but remain constrained by their limited input length (typically 512 tokens) and relatively shallow semantic understanding. To address these limitations, we propose QwenCLIP, a vision-language framework that replaces CLIP’s text encoder with a large language model (LLM)-based embedding module (e.g., Qwen3-Embedding) and introduces learnable prompts to enhance cross-modal alignment. By leveraging the extended context window and richer representations of LLMs, QwenCLIP captures comprehensive medical semantics from long-form clinical text, substantially improving medical image-text alignment and downstream performance on radiology benchmarks. Our code is publicly available at https://github.com/Wxy-24/QwenCLIP.

Method: Hybrid architecture combining EfficientNet and VGG19 with custom Pseudo-Newton Boosting layer for feature weight optimization and Sparsity-Induced Feature Reduction layer for redundancy removal.

Result: Achieved precision 0.9, recall 0.861, F1 score 0.88, loss 0.18, and accuracy 88.1%, outperforming baseline EfficientNet model.

Conclusion: The novel multi-tiered framework effectively addresses traditional transfer learning constraints in medical imaging and contributes to automated diagnostic tool development.

Abstract: This paper proposes a new enhanced model architecture to perform classification of lumbar spine degeneration with DICOM images while using a hybrid approach, integrating EfficientNet and VGG19 together with custom-designed components. The proposed model is differentiated from traditional transfer learning methods as it incorporates a Pseudo-Newton Boosting layer along with a Sparsity-Induced Feature Reduction Layer that forms a multi-tiered framework, further improving feature selection and representation. The Pseudo-Newton Boosting layer makes smart variations of feature weights, with more detailed anatomical features, which are mostly left out in a transfer learning setup. In addition, the Sparsity-Induced Layer removes redundancy for learned features, producing lean yet robust representations for pathology in the lumbar spine. This architecture is novel as it overcomes the constraints in the traditional transfer learning approach, especially in the high-dimensional context of medical images, and achieves a significant performance boost, reaching a precision of 0.9, recall of 0.861, F1 score of 0.88, loss of 0.18, and an accuracy of 88.1%, compared to the baseline model, EfficientNet. This work will present the architectures, preprocessing pipeline, and experimental results. The results contribute to the development of automated diagnostic tools for medical images.

Method: Leverage VLMs as semantic enhancers to enrich vision-based benchmarks with structured scene descriptions, then use a multi-modal interactive architecture that fuses visual and linguistic features, plus a post-hoc refinement module for prediction reliability.

Result: Extensive experiments on two autonomous driving benchmarks demonstrate state-of-the-art performance.

Conclusion: The approach offers a promising direction for integrating VLMs into reliable and interpretable autonomous driving systems by shifting their role from direct decision generators to semantic enhancers.

Abstract: Recent advancements in autonomous driving (AD) have explored the use of vision-language models (VLMs) within visual question answering (VQA) frameworks for direct driving decision-making. However, these approaches often depend on handcrafted prompts and suffer from inconsistent performance, limiting their robustness and generalization in real-world scenarios. In this work, we evaluate state-of-the-art open-source VLMs on high-level decision-making tasks using ego-view visual inputs and identify critical limitations in their ability to deliver reliable, context-aware decisions. Motivated by these observations, we propose a new approach that shifts the role of VLMs from direct decision generators to semantic enhancers. Specifically, we leverage their strong general scene understanding to enrich existing vision-based benchmarks with structured, linguistically rich scene descriptions. Building on this enriched representation, we introduce a multi-modal interactive architecture that fuses visual and linguistic features for more accurate decision-making and interpretable textual explanations. Furthermore, we design a post-hoc refinement module that utilizes VLMs to enhance prediction reliability. Extensive experiments on two autonomous driving benchmarks demonstrate that our approach achieves state-of-the-art performance, offering a promising direction for integrating VLMs into reliable and interpretable AD systems.

Method: Analyzed scaling laws across 15 semantic tasks and 4 imaging modalities. Evaluated deformation-guided augmentation strategies (random elastic and registration-guided deformation). Proposed a novel scalable augmentation approach generating diffeomorphic mappings from geodesic subspace using image registration.

Result: Larger datasets significantly improve segmentation performance following scaling trends. Both registered and generated deformation-based augmentation enhance data utilization efficiency. The proposed generated deformation method achieves superior performance and accelerated convergence, surpassing standard power law scaling without additional data.

Conclusion: This work provides insights into segmentation scalability and topological variation impact in medical imaging, enabling more efficient model development with reduced annotation and computational costs.

Abstract: The population loss of trained deep neural networks often exhibits power law scaling with the size of the training dataset, guiding significant performance advancements in deep learning applications. In this study, we focus on the scaling relationship with data size in the context of medical anatomical segmentation, a domain that remains underexplored. We analyze scaling laws for anatomical segmentation across 15 semantic tasks and 4 imaging modalities, demonstrating that larger datasets significantly improve segmentation performance, following similar scaling trends. Motivated by the topological isomorphism in images sharing anatomical structures, we evaluate the impact of deformation-guided augmentation strategies on data scaling laws, specifically random elastic deformation and registration-guided deformation. We also propose a novel, scalable image augmentation approach that generates diffeomorphic mappings from geodesic subspace based on image registration to introduce realistic deformation. Our experimental results demonstrate that both registered and generated deformation-based augmentation considerably enhance data utilization efficiency. The proposed generated deformation method notably achieves superior performance and accelerated convergence, surpassing standard power law scaling trends without requiring additional data. Overall, this work provides insights into the understanding of segmentation scalability and topological variation impact in medical imaging, thereby leading to more efficient model development with reduced annotation and computational costs.

Method: Built on Hema-Former, a multimodal module that bridges visual and textual representations at hierarchy level for different tasks (detection, classification, segmentation, morphology, mask language modeling, visual question answering) at different granularity, trained on 46 public datasets with over 700K images and 21K question-answer pairs.

Result: Uni-Hema achieves comparable or superior performance to single-task, single-dataset models across diverse hematological tasks while providing interpretable, morphologically relevant insights at the single-cell level.

Conclusion: The framework establishes a new standard for multi-task and multi-modal digital hematopathology, demonstrating unified reasoning capabilities across complex hematological conditions.

Abstract: Digital hematopathology requires cell-level analysis across diverse disease categories, including malignant disorders (e.g., leukemia), infectious conditions (e.g., malaria), and non-malignant red blood cell disorders (e.g., sickle cell disease). Whether single-task, vision-language, WSI-optimized, or single-cell hematology models, these approaches share a key limitation, they cannot provide unified, multi-task, multi-modal reasoning across the complexities of digital hematopathology. To overcome these limitations, we propose Uni-Hema, a multi-task, unified model for digital hematopathology integrating detection, classification, segmentation, morphology prediction, and reasoning across multiple diseases. Uni-Hema leverages 46 publicly available datasets, encompassing over 700K images and 21K question-answer pairs, and is built upon Hema-Former, a multimodal module that bridges visual and textual representations at the hierarchy level for the different tasks (detection, classification, segmentation, morphology, mask language modeling and visual question answer) at different granularity. Extensive experiments demonstrate that Uni-Hema achieves comparable or superior performance to train on a single-task and single dataset models, across diverse hematological tasks, while providing interpretable, morphologically relevant insights at the single-cell level. Our framework establishes a new standard for multi-task and multi-modal digital hematopathology. The code will be made publicly available.

Method: Uses Vision Language Models to generate noisy labels, then employs teacher-student model where teacher learns from VLM labels and student learns via weak supervision with noise-aware loss function.

Result: Experimental results show superior performance compared to VLMs and label model alone when using weak supervision to train student model.

Conclusion: The approach successfully addresses ephemeral gully detection challenges by reducing manual labeling effort through VLM-based weak supervision and provides the first public dataset for this task.

Abstract: Among soil erosion problems, Ephemeral Gullies are one of the most concerning phenomena occurring in agricultural fields. Their short temporal cycles increase the difficulty in automatically detecting them using classical computer vision approaches and remote sensing. Also, due to scarcity of and the difficulty in producing accurate labeled data, automatic detection of ephemeral gullies using Machine Learning is limited to zero-shot approaches which are hard to implement. To overcome these challenges, we present the first weakly supervised pipeline for detection of ephemeral gullies. Our method relies on remote sensing and uses Vision Language Models (VLMs) to drastically reduce the labor-intensive task of manual labeling. In order to achieve that, the method exploits: 1) the knowledge embedded in the VLM’s pretraining; 2) a teacher-student model where the teacher learns from noisy labels coming from the VLMs, and the student learns by weak supervision using teacher-generate labels and a noise-aware loss function. We also make available the first-of-its-kind dataset for semi-supervised detection of ephemeral gully from remote-sensed images. The dataset consists of a number of locations labeled by a group of soil and plant scientists, as well as a large number of unlabeled locations. The dataset represent more than 18,000 high-resolution remote-sensing images obtained over the course of 13 years. Our experimental results demonstrate the validity of our approach by showing superior performances compared to VLMs and the label model itself when using weak supervision to train an student model. The code and dataset for this work are made publicly available.

Method: Extract motion vectors from compressed video streams (H.264/HEVC), compute statistical divergences between real and generated videos, and explore MV-RGB fusion techniques.

Result: Identified systematic motion discrepancies in generated videos, with Pika and SVD closest to real motion. MV fusion improved classification accuracy up to 99.0% for real-vs-generated discrimination.

Conclusion: Compressed-domain motion vectors provide effective temporal signals for diagnosing motion defects and enhancing temporal reasoning in video analysis.

Abstract: Temporal realism remains a central weakness of current generative video models, as most evaluation metrics prioritize spatial appearance and offer limited sensitivity to motion. We introduce a scalable, model-agnostic framework that assesses temporal behavior using motion vectors (MVs) extracted directly from compressed video streams. Codec-generated MVs from standards such as H.264 and HEVC provide lightweight, resolution-consistent descriptors of motion dynamics. We quantify realism by computing Kullback-Leibler, Jensen-Shannon, and Wasserstein divergences between MV statistics of real and generated videos. Experiments on the GenVidBench dataset containing videos from eight state-of-the-art generators reveal systematic discrepancies from real motion: entropy-based divergences rank Pika and SVD as closest to real videos, MV-sum statistics favor VC2 and Text2Video-Zero, and CogVideo shows the largest deviations across both measures. Visualizations of MV fields and class-conditional motion heatmaps further reveal center bias, sparse and piecewise constant flows, and grid-like artifacts that frame-level metrics do not capture. Beyond evaluation, we investigate MV-RGB fusion through channel concatenation, cross-attention, joint embedding, and a motion-aware fusion module. Incorporating MVs improves downstream classification across ResNet, I3D, and TSN backbones, with ResNet-18 and ResNet-34 reaching up to 97.4% accuracy and I3D achieving 99.0% accuracy on real-versus-generated discrimination. These findings demonstrate that compressed-domain MVs provide an effective temporal signal for diagnosing motion defects in generative videos and for strengthening temporal reasoning in discriminative models. The implementation is available at: https://github.com/KurbanIntelligenceLab/Motion-Vector-Learning

Method: Distributed edge-central architecture: edge devices process video streams (detection, tracking, geo-mapping, feature extraction) and send lightweight metadata to central workstation, which performs cross-camera association using self-supervised camera link model with spatial-temporal constraints.

Result: Achieves real-time operation on 2K 15 FPS video streams with IDF1 score of 61.2 on RoundaboutHD dataset.

Conclusion: SAE-MCVT is the first scalable real-time MCVT framework suitable for city-scale deployment, addressing the gap between accuracy and practical performance requirements.

Abstract: In modern Intelligent Transportation Systems (ITS), cameras are a key component due to their ability to provide valuable information for multiple stakeholders. A central task is Multi-Camera Vehicle Tracking (MCVT), which generates vehicle trajectories and enables applications such as anomaly detection, traffic density estimation, and suspect vehicle tracking. However, most existing studies on MCVT emphasize accuracy while overlooking real-time performance and scalability. These two aspects are essential for real-world deployment and become increasingly challenging in city-scale applications as the number of cameras grows. To address this issue, we propose SAE-MCVT, the first scalable real-time MCVT framework. The system includes several edge devices that interact with one central workstation separately. On the edge side, live RTSP video streams are serialized and processed through modules including object detection, object tracking, geo-mapping, and feature extraction. Only lightweight metadata – vehicle locations and deep appearance features – are transmitted to the central workstation. On the central side, cross-camera association is calculated under the constraint of spatial-temporal relations between adjacent cameras, which are learned through a self-supervised camera link model. Experiments on the RoundaboutHD dataset show that SAE-MCVT maintains real-time operation on 2K 15 FPS video streams and achieves an IDF1 score of 61.2. To the best of our knowledge, this is the first scalable real-time MCVT framework suitable for city-scale deployment.

Method: Curated a pilot dataset of images depicting traffic signs and road-safety norms from school textbooks, evaluated models in zero-shot setting.

Result: Models struggle with safety reasoning, showing performance gaps between human learning and model interpretation.

Conclusion: There are significant gaps in multi-modal LLMs’ understanding of road safety concepts that need to be addressed for future autonomous vehicle systems.

Abstract: Following road safety norms is non-negotiable not only for humans but also for the AI systems that govern autonomous vehicles. In this work, we evaluate how well multi-modal large language models (LLMs) understand road safety concepts, specifically through schematic and illustrative representations. We curate a pilot dataset of images depicting traffic signs and road-safety norms sourced from school text books and use it to evaluate models capabilities in a zero-shot setting. Our preliminary results show that these models struggle with safety reasoning and reveal gaps between human learning and model interpretation. We further provide an analysis of these performance gaps for future research.

Method: Curriculum-based Relative Policy Optimization (CuRPO) uses CoT length and generalized IoU rewards as complexity indicators to structure training data from simpler to more challenging examples.

Result: CuRPO outperforms existing methods including Visual-RFT with up to +12.52 mAP improvement on RefCOCO, and shows exceptional efficiency and robustness in few-shot learning scenarios.

Conclusion: CuRPO effectively addresses the limitations of RL-based CoT fine-tuning in visual grounding by progressively structuring training complexity, delivering strong performance especially for ambiguous and intricate textual descriptions.

Abstract: Chain-of-Thought (CoT) prompting has recently shown significant promise across various NLP and computer vision tasks by explicitly generating intermediate reasoning steps. However, we find that reinforcement learning (RL)-based fine-tuned CoT reasoning can paradoxically degrade performance in Visual Grounding tasks, particularly as CoT outputs become lengthy or complex. Additionally, our analysis reveals that increased dataset size does not always enhance performance due to varying data complexities. Motivated by these findings, we propose Curriculum-based Relative Policy Optimization (CuRPO), a novel training strategy that leverages CoT length and generalized Intersection over Union (gIoU) rewards as complexity indicators to progressively structure training data from simpler to more challenging examples. Extensive experiments on RefCOCO, RefCOCO+, RefCOCOg, and LISA datasets demonstrate the effectiveness of our approach. CuRPO consistently outperforms existing methods, including Visual-RFT, with notable improvements of up to +12.52 mAP on RefCOCO. Moreover, CuRPO exhibits exceptional efficiency and robustness, delivering strong localization performance even in few-shot learning scenarios, particularly benefiting tasks characterized by ambiguous and intricate textual descriptions.The code is released on https://github.com/qyoung-yan/CuRPO.

Method: Group visually similar frames into clusters before splitting datasets into training, validation, and test sets to ensure balanced and representative partitions.

Result: Produces more representative, balanced, and reliable dataset partitions that prevent information leakage.

Conclusion: The cluster-based frame selection strategy effectively addresses information leakage issues in video datasets, leading to more trustworthy model evaluation.

Abstract: We propose a cluster-based frame selection strategy to mitigate information leakage in video-derived frames datasets. By grouping visually similar frames before splitting into training, validation, and test sets, the method produces more representative, balanced, and reliable dataset partitions.

Method: Generate procedurally-generated data using simple algorithms like formal grammars, pretrain ViTs on this data while bypassing visual patch embedding mechanisms, then follow with standard image-based training.

Result: Significant improvements in data efficiency, convergence speed, and downstream performance - 1% procedural data in warm-up phase improves ImageNet-1k accuracy by over 1.7%, equivalent to 28% of ImageNet-1k data.

Conclusion: Procedural data pretraining offers a promising path toward data-efficient and domain-agnostic pretraining strategies for vision transformers.

Abstract: Transformers show remarkable versatility across domains, suggesting the existence of inductive biases beneficial across modalities. In this work, we explore a new way to instil such generic biases in vision transformers (ViTs) by pretraining on procedurally-generated data devoid of visual or semantic content. We generate this data with simple algorithms such as formal grammars, so the results bear no relationship to either natural or synthetic images. We use this procedurally-generated data to pretrain ViTs in a warm-up phase that bypasses their visual patch embedding mechanisms, thus encouraging the models to internalise abstract computational priors. When followed by standard image-based training, this warm-up significantly improves data efficiency, convergence speed, and downstream performance. On ImageNet-1k for example, allocating just 1% of the training budget to procedural data improves final accuracy by over 1.7%. In terms of its effect on performance, 1% procedurally generated data is thus equivalent to 28% of the ImageNet-1k data. These findings suggest a promising path toward new data-efficient and domain-agnostic pretraining strategies.

Method: Learn continuous scalar fields on image domains parameterized by trained networks, where fields are solutions to Poisson PDE and heat diffusion equations. Segmentation is performed using watershed method on these learned fields.

Result: Competitive results on public datasets showing excellent cell segmentation performance with simplified implementation, reduced training/inference times, lower energy consumption, and small memory footprint.

Conclusion: The proposed simple yet geometrically insightful approach using scalar fields and watershed segmentation achieves robust cell segmentation with computational advantages, making it suitable for edge computing environments.

Abstract: We investigate image segmentation of cells under the lens of scalar fields. Our goal is to learn a continuous scalar field on image domains such that its segmentation produces robust instances for cells present in images. This field is a function parameterized by the trained network, and its segmentation is realized by the watershed method. The fields we experiment with are solutions to the Poisson partial differential equation and a diffusion mimicking the steady-state solution of the heat equation. These solutions are obtained by minimizing just the field residuals, no regularization is needed, providing a robust regression capable of diminishing the adverse impacts of outliers in the training data and allowing for sharp cell boundaries. A single tensor is all that is needed to train a \unet\ thus simplifying implementation, lowering training and inference times, hence reducing energy consumption, and requiring a small memory footprint, all attractive features in edge computing. We present competitive results on public datasets from the literature and show that our novel, simple yet geometrically insightful approach can achieve excellent cell segmentation results.

Method: EchoAgent orchestrates specialized vision tools under LLM control with a key measurement-feasibility prediction model that determines whether anatomical structures are reliably measurable in each frame, enabling autonomous tool selection. Evaluated on curated benchmark of clinically validated video-query pairs.

Result: EchoAgent achieves accurate, interpretable results despite complexity of spatiotemporal video analysis, with outputs grounded in visual evidence and clinical guidelines for transparency and traceability.

Conclusion: Demonstrates feasibility of agentic, guideline-aligned reasoning for echocardiographic video analysis using task-specific tools and full video-level automation, setting new direction for trustworthy AI in cardiac ultrasound.

Abstract: Purpose: Echocardiographic interpretation requires video-level reasoning and guideline-based measurement analysis, which current deep learning models for cardiac ultrasound do not support. We present EchoAgent, a framework that enables structured, interpretable automation for this domain. Methods: EchoAgent orchestrates specialized vision tools under Large Language Model (LLM) control to perform temporal localization, spatial measurement, and clinical interpretation. A key contribution is a measurement-feasibility prediction model that determines whether anatomical structures are reliably measurable in each frame, enabling autonomous tool selection. We curated a benchmark of diverse, clinically validated video-query pairs for evaluation. Results: EchoAgent achieves accurate, interpretable results despite added complexity of spatiotemporal video analysis. Outputs are grounded in visual evidence and clinical guidelines, supporting transparency and traceability. Conclusion: This work demonstrates the feasibility of agentic, guideline-aligned reasoning for echocardiographic video analysis, enabled by task-specific tools and full video-level automation. EchoAgent sets a new direction for trustworthy AI in cardiac ultrasound.

Method: Discovers skill-attributes that transcend sport boundaries (balance, control, hand positioning), then trains a multimodal language model to generate actionable feedback and proficiency levels for novel videos.

Result: Achieves gains up to 60% relative to state-of-the-art in both cross-sport (transfer) and intra-sport (in-domain) settings across multiple datasets.

Conclusion: By abstracting shared behaviors indicative of human skill, the proposed video representation generalizes substantially better than existing techniques and enriches multimodal large language models.

Abstract: Skill assessment from video entails rating the quality of a person’s physical performance and explaining what could be done better. Today’s models specialize for an individual sport, and suffer from the high cost and scarcity of expert-level supervision across the long tail of sports. Towards closing that gap, we explore transferable video representations for skill assessment. Our CrossTrainer approach discovers skill-attributes, such as balance, control, and hand positioning – whose meaning transcends the boundaries of any given sport, then trains a multimodal language model to generate actionable feedback for a novel video, e.g., “lift hands more to generate more power” as well as its proficiency level, e.g., early expert. We validate the new model on multiple datasets for both cross-sport (transfer) and intra-sport (in-domain) settings, where it achieves gains up to 60% relative to the state of the art. By abstracting out the shared behaviors indicative of human skill, the proposed video representation generalizes substantially better than an array of existing techniques, enriching today’s multimodal large language models.

Method: Uses iterative convolutional dictionary feature decoupling (CD-FDM) to separate features into cross-contrast and intra-contrast components, and dual-prompt feature fusion expert module (DP-FFEM) with frequency and adaptive routing prompts for optimal feature fusion.

Result: Outperforms state-of-the-art methods on public multi-contrast MRI datasets in reconstructing fine details and demonstrates strong generalization on unseen datasets.

Conclusion: CD-DPE effectively addresses contrast disparities in multi-contrast MRI super-resolution through feature decoupling and intelligent fusion, achieving superior reconstruction quality and generalization.

Abstract: Multi-contrast magnetic resonance imaging (MRI) super-resolution intends to reconstruct high-resolution (HR) images from low-resolution (LR) scans by leveraging structural information present in HR reference images acquired with different contrasts. This technique enhances anatomical detail and soft tissue differentiation, which is vital for early diagnosis and clinical decision-making. However, inherent contrasts disparities between modalities pose fundamental challenges in effectively utilizing reference image textures to guide target image reconstruction, often resulting in suboptimal feature integration. To address this issue, we propose a dual-prompt expert network based on a convolutional dictionary feature decoupling (CD-DPE) strategy for multi-contrast MRI super-resolution. Specifically, we introduce an iterative convolutional dictionary feature decoupling module (CD-FDM) to separate features into cross-contrast and intra-contrast components, thereby reducing redundancy and interference. To fully integrate these features, a novel dual-prompt feature fusion expert module (DP-FFEM) is proposed. This module uses a frequency prompt to guide the selection of relevant reference features for incorporation into the target image, while an adaptive routing prompt determines the optimal method for fusing reference and target features to enhance reconstruction quality. Extensive experiments on public multi-contrast MRI datasets demonstrate that CD-DPE outperforms state-of-the-art methods in reconstructing fine details. Additionally, experiments on unseen datasets demonstrated that CD-DPE exhibits strong generalization capabilities.

Method: Uses Bi-Angular Multipath Enhancement to model Angle-of-Arrival and Angle-of-Departure to recover secondary reflections, and a simulation-to-reality Hierarchical Diffusion framework to transform radar responses into complete scene understanding.

Result: Reduces Chamfer Distance by 60% (down to 16 cm) compared to state-of-the-art in layout reconstruction, and achieves 58% IoU for mmWave-based object detection - the first such system.

Conclusion: RISE establishes a new foundation for geometry-aware and privacy-preserving indoor scene understanding using a single static radar.

Abstract: Robust and privacy-preserving indoor scene understanding remains a fundamental open problem. While optical sensors such as RGB and LiDAR offer high spatial fidelity, they suffer from severe occlusions and introduce privacy risks in indoor environments. In contrast, millimeter-wave (mmWave) radar preserves privacy and penetrates obstacles, but its inherently low spatial resolution makes reliable geometric reasoning difficult. We introduce RISE, the first benchmark and system for single-static-radar indoor scene understanding, jointly targeting layout reconstruction and object detection. RISE is built upon the key insight that multipath reflections, traditionally treated as noise, encode rich geometric cues. To exploit this, we propose a Bi-Angular Multipath Enhancement that explicitly models Angle-of-Arrival and Angle-of-Departure to recover secondary (ghost) reflections and reveal invisible structures. On top of these enhanced observations, a simulation-to-reality Hierarchical Diffusion framework transforms fragmented radar responses into complete layout reconstruction and object detection. Our benchmark contains 50,000 frames collected across 100 real indoor trajectories, forming the first large-scale dataset dedicated to radar-based indoor scene understanding. Extensive experiments show that RISE reduces the Chamfer Distance by 60% (down to 16 cm) compared to the state of the art in layout reconstruction, and delivers the first mmWave-based object detection, achieving 58% IoU. These results establish RISE as a new foundation for geometry-aware and privacy-preserving indoor scene understanding using a single static radar.

Method: Developed a 2.5D context-aware model that leverages multi-slice information to avoid ambiguity from isolated slices. Trained on both 3D slice sequences and static 2D images, with a gated strategy that selectively uses metadata-enhanced predictions based on uncertainty scores.

Result: 2.5D method achieved 99.49% accuracy (vs 98.74% for 2D reference), reducing errors by 60%. In brain tumor detection, metadata-enhanced predictions boosted accuracy from 97.0% to 98.0%, reducing misdiagnoses by 33.3%.

Conclusion: 2.5D context is crucial for accurate plane orientation identification. The generated metadata significantly improves diagnostic tasks, and the model has been integrated into an open-source web application.

Abstract: Humans can easily identify anatomical planes (axial, coronal, and sagittal) on a 2D MRI slice, but automated systems struggle with this task. Missing plane orientation metadata can complicate analysis, increase domain shift when merging heterogeneous datasets, and reduce accuracy of diagnostic classifiers. This study develops a classifier that accurately generates plane orientation metadata. We adopt a 2.5D context-aware model that leverages multi-slice information to avoid ambiguity from isolated slices and enable robust feature learning. We train the 2.5D model on both 3D slice sequences and static 2D images. While our 2D reference model achieves 98.74% accuracy, our 2.5D method raises this to 99.49%, reducing errors by 60%, highlighting the importance of 2.5D context. We validate the utility of our generated metadata in a brain tumor detection task. A gated strategy selectively uses metadata-enhanced predictions based on uncertainty scores, boosting accuracy from 97.0% with an image-only model to 98.0%, reducing misdiagnoses by 33.3%. We integrate our plane orientation model into an interactive web application and provide it open-source.

Method: LINGUAL receives natural language instructions from experts, translates them into executable programs through in-context learning, and automatically performs corresponding sub-tasks without human intervention.

Result: LINGUAL achieves comparable or superior performance to active learning baselines in active domain adaptation while reducing estimated annotation time by approximately 80%.

Conclusion: Language-guided frameworks like LINGUAL provide an effective alternative to traditional active learning, significantly reducing expert annotation effort while maintaining or improving segmentation performance.

Abstract: Although active learning (AL) in segmentation tasks enables experts to annotate selected regions of interest (ROIs) instead of entire images, it remains highly challenging, labor-intensive, and cognitively demanding due to the blurry and ambiguous boundaries commonly observed in medical images. Also, in conventional AL, annotation effort is a function of the ROI- larger regions make the task cognitively easier but incur higher annotation costs, whereas smaller regions demand finer precision and more attention from the expert. In this context, language guidance provides an effective alternative, requiring minimal expert effort while bypassing the cognitively demanding task of precise boundary delineation in segmentation. Towards this goal, we introduce LINGUAL: a framework that receives natural language instructions from an expert, translates them into executable programs through in-context learning, and automatically performs the corresponding sequence of sub-tasks without any human intervention. We demonstrate the effectiveness of LINGUAL in active domain adaptation (ADA) achieving comparable or superior performance to AL baselines while reducing estimated annotation time by approximately 80%.

Method: WaRPAD leverages self-supervised models pre-trained with augmentations like RandomResizedCrop. It quantifies sensitivity of image embeddings to high-frequency perturbations via Haar wavelet decomposition, rescales images to multiples of model input size, divides into patches, and computes detection scores by averaging patch scores.

Result: The method achieves competitive performance on real datasets of diverse resolutions/domains and images from 23 different generative models, showing strong robustness to test-time corruptions and applicability across various self-supervised models.

Conclusion: WaRPAD provides an effective training-free approach for AI-generated image detection that leverages inherent properties of self-supervised models and demonstrates broad applicability across different models and domains.

Abstract: AI-generated image detection has become crucial with the rapid advancement of vision-generative models. Instead of training detectors tailored to specific datasets, we study a training-free approach leveraging self-supervised models without requiring prior data knowledge. These models, pre-trained with augmentations like RandomResizedCrop, learn to produce consistent representations across varying resolutions. Motivated by this, we propose WaRPAD, a training-free AI-generated image detection algorithm based on self-supervised models. Since neighborhood pixel differences in images are highly sensitive to resizing operations, WaRPAD first defines a base score function that quantifies the sensitivity of image embeddings to perturbations along high-frequency directions extracted via Haar wavelet decomposition. To simulate robustness against cropping augmentation, we rescale each image to a multiple of the models input size, divide it into smaller patches, and compute the base score for each patch. The final detection score is then obtained by averaging the scores across all patches. We validate WaRPAD on real datasets of diverse resolutions and domains, and images generated by 23 different generative models. Our method consistently achieves competitive performance and demonstrates strong robustness to test-time corruptions. Furthermore, as invariance to RandomResizedCrop is a common training scheme across self-supervised models, we show that WaRPAD is applicable across self-supervised models.

Method: Proposes a deformation-free framework that directly out-paints segmented garments, and introduces region-adaptive decoupled attention (RADA) with chained mask injection to control fine-grained attributes.

Result: Achieves high-quality fashion showcase generation with faithful garment preservation and enhanced controllability, outperforming state-of-the-art methods in experiments.

Conclusion: FashionMAC successfully addresses key challenges in garment-centric image generation by eliminating deformation and enabling precise appearance control through innovative attention mechanisms.

Abstract: Garment-centric fashion image generation aims to synthesize realistic and controllable human models dressing a given garment, which has attracted growing interest due to its practical applications in e-commerce. The key challenges of the task lie in two aspects: (1) faithfully preserving the garment details, and (2) gaining fine-grained controllability over the model’s appearance. Existing methods typically require performing garment deformation in the generation process, which often leads to garment texture distortions. Also, they fail to control the fine-grained attributes of the generated models, due to the lack of specifically designed mechanisms. To address these issues, we propose FashionMAC, a novel diffusion-based deformation-free framework that achieves high-quality and controllable fashion showcase image generation. The core idea of our framework is to eliminate the need for performing garment deformation and directly outpaint the garment segmented from a dressed person, which enables faithful preservation of the intricate garment details. Moreover, we propose a novel region-adaptive decoupled attention (RADA) mechanism along with a chained mask injection strategy to achieve fine-grained appearance controllability over the synthesized human models. Specifically, RADA adaptively predicts the generated regions for each fine-grained text attribute and enforces the text attribute to focus on the predicted regions by a chained mask injection strategy, significantly enhancing the visual fidelity and the controllability. Extensive experiments validate the superior performance of our framework compared to existing state-of-the-art methods.

Method: Proposed a novel approach using latent diffusion models to perform super-resolution on coarse-grid flood maps, incorporating physics-informed inputs to enhance interpretability and address black-box limitations.

Result: Latent diffusion models substantially decrease computational time while maintaining accuracy comparable to fine-grid flood maps. They exhibit superior generalizability across different locations, with transfer learning accelerating adaptation to new geographic regions.

Conclusion: The approach enables real-time flood risk management by producing high-fidelity flood maps efficiently, with improved generalizability and interpretability through physics-informed inputs.

Abstract: Flood prediction is critical for emergency planning and response to mitigate human and economic losses. Traditional physics-based hydrodynamic models generate high-resolution flood maps using numerical methods requiring fine-grid discretization; which are computationally intensive and impractical for real-time large-scale applications. While recent studies have applied convolutional neural networks for flood map super-resolution with good accuracy and speed, they suffer from limited generalizability to unseen areas. In this paper, we propose a novel approach that leverages latent diffusion models to perform super-resolution on coarse-grid flood maps, with the objective of achieving the accuracy of fine-grid flood maps while significantly reducing inference time. Experimental results demonstrate that latent diffusion models substantially decrease the computational time required to produce high-fidelity flood maps without compromising on accuracy, enabling their use in real-time flood risk management. Moreover, diffusion models exhibit superior generalizability across different physical locations, with transfer learning further accelerating adaptation to new geographic regions. Our approach also incorporates physics-informed inputs, addressing the common limitation of black-box behavior in machine learning, thereby enhancing interpretability. Code is available at https://github.com/neosunhan/flood-diff.

Method: Two-stage framework: 1) Saliency-based defect region proposals using local discontinuities, 2) YOLOX-based deep learning detector on saliency-enhanced images with bounding-box level brightness augmentation.

Result: Experimental results on standard datasets confirm good performance in terms of accuracy and computational efficiency.

Conclusion: The framework shows potential for implementation in self-powered inspection systems for bridge defect detection.

Abstract: Anomaly object detection and classification are one of the main challenging tasks in computer vision and pattern recognition. In this paper, we propose a new method to automatically detect, localize and classify defects in concrete bridge structures using drone imagery. This framework is constituted of two main stages. The first stage uses saliency for defect region proposals where defects often exhibit local discontinuities in the normal surface patterns with regard to their surrounding. The second stage employs a YOLOX-based deep learning detector that operates on saliency-enhanced images obtained by applying bounding-box level brightness augmentation to salient defect regions. Experimental results on standard datasets confirm the performance of our framework and its suitability in terms of accuracy and computational efficiency, which give a huge potential to be implemented in a self-powered inspection system.

Method: Introduces Compressed Semantic Prefilling to encode high-level semantics into efficient prefixes, and Semantic Alignment Guidance to align visual hidden states with target semantics during decoding.

Result: Achieves superior visual fidelity and semantic alignment on instruction editing and controllable generation benchmarks, outperforming prior AR-based methods.

Conclusion: SCAR effectively addresses conditioning limitations in AR models for image editing while maintaining controllability and generalizing across different AR paradigms.

Abstract: Recently, autoregressive (AR) models have shown strong potential in image generation, offering better scalability and easier integration with unified multi-modal systems compared to diffusion-based methods. However, extending AR models to general image editing remains challenging due to weak and inefficient conditioning, often leading to poor instruction adherence and visual artifacts. To address this, we propose SCAR, a Semantic-Context-driven method for Autoregressive models. SCAR introduces two key components: Compressed Semantic Prefilling, which encodes high-level semantics into a compact and efficient prefix, and Semantic Alignment Guidance, which aligns the last visual hidden states with target semantics during autoregressive decoding to enhance instruction fidelity. Unlike decoding-stage injection methods, SCAR builds upon the flexibility and generality of vector-quantized-based prefilling while overcoming its semantic limitations and high cost. It generalizes across both next-token and next-set AR paradigms with minimal architectural changes. SCAR achieves superior visual fidelity and semantic alignment on both instruction editing and controllable generation benchmarks, outperforming prior AR-based methods while maintaining controllability. All code will be released.

Method: Uses an efficient segmentation decoder to generate object masks as semantic prior, merges visual tokens into compact object-centric representations, and employs centroid-guided sorting to restore spatial coherence.

Result: Achieves SOTA on six benchmarks for fixed-rate compression, dramatic efficiency gains in adaptive-rate settings, and maintains 97.4% baseline performance with only 2.2% of visual tokens.

Conclusion: Object-centric representations are superior for efficient and effective LVLM processing, demonstrating the value of semantic guidance in visual token compression.

Abstract: Large Vision-Language Models (LVLMs) usually suffer from prohibitive computational and memory costs due to the quadratic growth of visual tokens with image resolution. Existing token compression methods, while varied, often lack a high-level semantic understanding, leading to suboptimal merges, information redundancy, or context loss. To address these limitations, we introduce CORE (Compact Object-centric REpresentations), a new paradigm for visual token compression. CORE leverages an efficient segmentation decoder to generate object masks, which serve as a high-level semantic prior to guide the merging of visual tokens into a compact set of object-centric representations. Furthermore, a novel centroid-guided sorting mechanism restores a coherent spatial order to the merged tokens, preserving vital positional information. Extensive experiments show that CORE not only establishes a new state-of-the-art on six authoritative benchmarks for fixed-rate compression, but also achieves dramatic efficiency gains in adaptive-rate settings. Even under extreme compression, after aggressively retaining with only 2.2% of all visual tokens, CORE still maintains 97.4% of baseline performance. Our work demonstrates the superiority of object-centric representations for efficient and effective LVLM processing.

Method: Leverages pre-trained generative engine to synthesize complete classifier parameters from minimal task information (single image + clinical text description), generating weights for lightweight classifiers like EfficientNet-V2.

Result: Achieves state-of-the-art performance on ISIC 2018 skin lesion and rare disease datasets, significantly outperforming few-shot and zero-shot methods in 1-shot and 5-shot regimes.

Conclusion: Enables rapid development of AI diagnostic tools for rare diseases with limited data, paving way for practical deployment in data-scarce medical scenarios.

Abstract: Deep learning models have achieved remarkable success in medical image analysis but are fundamentally constrained by the requirement for large-scale, meticulously annotated datasets. This dependency on “big data” is a critical bottleneck in the medical domain, where patient data is inherently difficult to acquire and expert annotation is expensive, particularly for rare diseases where samples are scarce by definition. To overcome this fundamental challenge, we propose a novel paradigm: Zero-Training Task-Specific Model Synthesis (ZS-TMS). Instead of adapting a pre-existing model or training a new one, our approach leverages a large-scale, pre-trained generative engine to directly synthesize the entire set of parameters for a task-specific classifier. Our framework, the Semantic-Guided Parameter Synthesizer (SGPS), takes as input minimal, multi-modal task information as little as a single example image (1-shot) and a corresponding clinical text description to directly synthesize the entire set of parameters for a task-specific classifier. The generative engine interprets these inputs to generate the weights for a lightweight, efficient classifier (e.g., an EfficientNet-V2), which can be deployed for inference immediately without any task-specific training or fine-tuning. We conduct extensive evaluations on challenging few-shot classification benchmarks derived from the ISIC 2018 skin lesion dataset and a custom rare disease dataset. Our results demonstrate that SGPS establishes a new state-of-the-art, significantly outperforming advanced few-shot and zero-shot learning methods, especially in the ultra-low data regimes of 1-shot and 5-shot classification. This work paves the way for the rapid development and deployment of AI-powered diagnostic tools, particularly for the long tail of rare diseases where data is critically limited.

Method: Multi-stage deep learning pipeline with three main stages: (1) U-Net for automatic glenoid and humerus segmentation, (2) network for glenoid rim point detection, and (3) PCA, projection, and circle fitting for geometric bone loss calculation.

Result: Automated measurements showed strong agreement with consensus readings (ICC 0.84 vs 0.78 for surgeons), performed well in low- and high-bone-loss subgroups, and achieved good classification recall (0.714 for low, 0.857 for high severity cases).

Conclusion: The method provides a time-efficient and clinically reliable tool for preoperative planning in shoulder instability and screening patients with substantial glenoid bone loss.

Abstract: Reliable measurement of glenoid bone loss is essential for operative planning in shoulder instability, but current manual and semi-automated methods are time-consuming and often subject to interreader variability. We developed and validated a fully automated deep learning pipeline for measuring glenoid bone loss on three-dimensional computed tomography (CT) scans using a linear-based, en-face view, best-circle method. Shoulder CT images of 91 patients (average age, 40 years; range, 14-89 years; 65 men) were retrospectively collected along with manual labels including glenoid segmentation, landmarks, and bone loss measurements. The multi-stage algorithm has three main stages: (1) segmentation, where we developed a U-Net to automatically segment the glenoid and humerus; (2) anatomical landmark detection, where a second network predicts glenoid rim points; and (3) geometric fitting, where we applied principal component analysis (PCA), projection, and circle fitting to compute the percentage of bone loss. The automated measurements showed strong agreement with consensus readings and exceeded surgeon-to-surgeon consistency (intraclass correlation coefficient (ICC) 0.84 vs 0.78), including in low- and high-bone-loss subgroups (ICC 0.71 vs 0.63 and 0.83 vs 0.21, respectively; P < 0.001). For classifying patients into low, medium, and high bone-loss categories, the pipeline achieved a recall of 0.714 for low and 0.857 for high severity, with no low cases misclassified as high or vice versa. These results suggest that our method is a time-efficient and clinically reliable tool for preoperative planning in shoulder instability and for screening patients with substantial glenoid bone loss. Code and dataset are available at https://github.com/Edenliu1/Auto-Glenoid-Measurement-DL-Pipeline.

Method: DEER-3D follows a “Decompose, Diagnostic Evaluation, Edit, and Re-train” workflow that diagnoses predicate-level errors and executes minimal 3D scene edits (recoloring, repositioning) to produce targeted counterfactual supervision for iterative model fine-tuning.

Result: The framework consistently demonstrates improvements across multiple benchmarks for 3D grounding and scene understanding tasks through iterative refinement, significantly enhancing grounding accuracy.

Conclusion: Targeted, error-driven scene editing effectively bridges linguistic reasoning capabilities with spatial grounding in 3D LLMs, addressing specific model weaknesses without costly scene reconstruction or large-scale data collection.

Abstract: Despite recent progress in 3D-LLMs, they remain limited in accurately grounding language to visual and spatial elements in 3D environments. This limitation stems in part from training data that focuses on language reasoning rather than spatial understanding due to scarce 3D resources, leaving inherent grounding biases unresolved. To address this, we propose 3D scene editing as a key mechanism to generate precise visual counterfactuals that mitigate these biases through fine-grained spatial manipulation, without requiring costly scene reconstruction or large-scale 3D data collection. Furthermore, to make these edits targeted and directly address the specific weaknesses of the model, we introduce DEER-3D, an error-driven framework following a structured “Decompose, Diagnostic Evaluation, Edit, and Re-train” workflow, rather than broadly or randomly augmenting data as in conventional approaches. Specifically, upon identifying a grounding failure of the 3D-LLM, our framework first diagnoses the exact predicate-level error (e.g., attribute or spatial relation). It then executes minimal, predicate-aligned 3D scene edits, such as recoloring or repositioning, to produce targeted counterfactual supervision for iterative model fine-tuning, significantly enhancing grounding accuracy. We evaluate our editing pipeline across multiple benchmarks for 3D grounding and scene understanding tasks, consistently demonstrating improvements across all evaluated datasets through iterative refinement. DEER-3D underscores the effectiveness of targeted, error-driven scene editing in bridging linguistic reasoning capabilities with spatial grounding in 3D LLMs.

Method: Proposes GCA-ResUNet that integrates Grouped Coordinate Attention (GCA) into ResNet-50 residual blocks. GCA uses grouped coordinate modeling to jointly encode global dependencies across channels and spatial locations with minimal parameter and FLOP overhead.

Result: Achieves Dice scores of 86.11% on Synapse dataset and 92.64% on ACDC dataset, surpassing state-of-the-art baselines while maintaining fast inference and computational efficiency.

Conclusion: GCA provides a practical way to enhance convolutional architectures with global modeling capability, enabling high-accuracy and resource-efficient medical image segmentation.

Abstract: Medical image segmentation underpins computer-aided diagnosis and therapy by supporting clinical diagnosis, preoperative planning, and disease monitoring. While U-Net style convolutional neural networks perform well due to their encoder-decoder structures with skip connections, they struggle to capture long-range dependencies. Transformer-based variants address global context but often require heavy computation and large training datasets. This paper proposes GCA-ResUNet, an efficient segmentation network that integrates Grouped Coordinate Attention (GCA) into ResNet-50 residual blocks. GCA uses grouped coordinate modeling to jointly encode global dependencies across channels and spatial locations, strengthening feature representation and boundary delineation while adding minimal parameter and FLOP overhead compared with self-attention. On the Synapse dataset, GCA-ResUNet achieves a Dice score of 86.11%, and on the ACDC dataset, it reaches 92.64%, surpassing several state-of-the-art baselines while maintaining fast inference and favorable computational efficiency. These results indicate that GCA offers a practical way to enhance convolutional architectures with global modeling capability, enabling high-accuracy and resource-efficient medical image segmentation.

Method: Uses a fully transformer-based architecture with Swin-Transformer for joint feature encoding, grid-level sparse Mixture-of-Experts (GMoE) to capture inter-modal and intra-view dependencies, and an anchor-free transformer detection head for coordinate regression via heat-map supervision.

Result: Achieves leading performance on drone-to-satellite task with 87.51% accuracy at IoU=0.25 and 62.50% mIoU in test set, significantly outperforming DetGeo (61.97% and 57.66% respectively).

Conclusion: SMGeo demonstrates that end-to-end transformer architecture with specialized components like GMoE and anchor-free detection effectively addresses cross-view geo-localization challenges, achieving superior accuracy while supporting interactive use through click prompting.

Abstract: Cross-view object Geo-localization aims to precisely pinpoint the same object across large-scale satellite imagery based on drone images. Due to significant differences in viewpoint and scale, coupled with complex background interference, traditional multi-stage “retrieval-matching” pipelines are prone to cumulative errors. To address this, we present SMGeo, a promptable end-to-end transformer-based model for object Geo-localization. This model supports click prompting and can output object Geo-localization in real time when prompted to allow for interactive use. The model employs a fully transformer-based architecture, utilizing a Swin-Transformer for joint feature encoding of both drone and satellite imagery and an anchor-free transformer detection head for coordinate regression. In order to better capture both inter-modal and intra-view dependencies, we introduce a grid-level sparse Mixture-of-Experts (GMoE) into the cross-view encoder, allowing it to adaptively activate specialized experts according to the content, scale and source of each grid. We also employ an anchor-free detection head for coordinate regression, directly predicting object locations via heat-map supervision in the reference images. This approach avoids scale bias and matching complexity introduced by predefined anchor boxes. On the drone-to-satellite task, SMGeo achieves leading performance in accuracy at IoU=0.25 and mIoU metrics (e.g., 87.51%, 62.50%, and 61.45% in the test set, respectively), significantly outperforming representative methods such as DetGeo (61.97%, 57.66%, and 54.05%, respectively). Ablation studies demonstrate complementary gains from shared encoding, query-guided fusion, and grid-level sparse mixture-of-experts.

Method: Three BCE-based components: joint learning for feature and classifier optimization, contrastive learning for intra-class compactness, and uniform learning for balanced classifier separability.

Result: Achieves SOTA performance on multiple long-tailed datasets (CIFAR10-LT, CIFAR100-LT, ImageNet-LT, iNaturalist2018) with higher feature compactness/separability and balanced classifier separability.

Conclusion: BCE3S effectively addresses LTR challenges by decoupling metrics in multiple Sigmoid and synergistically combining three learning components to achieve superior performance.

Abstract: For long-tailed recognition (LTR) tasks, high intra-class compactness and inter-class separability in both head and tail classes, as well as balanced separability among all the classifier vectors, are preferred. The existing LTR methods based on cross-entropy (CE) loss not only struggle to learn features with desirable properties but also couple imbalanced classifier vectors in the denominator of its Softmax, amplifying the imbalance effects in LTR. In this paper, for the LTR, we propose a binary cross-entropy (BCE)-based tripartite synergistic learning, termed BCE3S, which consists of three components: (1) BCE-based joint learning optimizes both the classifier and sample features, which achieves better compactness and separability among features than the CE-based joint learning, by decoupling the metrics between feature and the imbalanced classifier vectors in multiple Sigmoid; (2) BCE-based contrastive learning further improves the intra-class compactness of features; (3) BCE-based uniform learning balances the separability among classifier vectors and interactively enhances the feature properties by combining with the joint learning. The extensive experiments show that the LTR model trained by BCE3S not only achieves higher compactness and separability among sample features, but also balances the classifier’s separability, achieving SOTA performance on various long-tailed datasets such as CIFAR10-LT, CIFAR100-LT, ImageNet-LT, and iNaturalist2018.

Method: Uses frozen MLLM as planner to analyze degraded images and generate frequency-aware restoration plans. These guide a LoRA-MoE module in diffusion-based executor that dynamically selects high/low-frequency experts using input image’s frequency features. Includes adversarial training and frequency regularization loss.

Result: Achieves state-of-the-art performance across seven restoration tasks and exhibits strong zero-shot generalization under mixed degradations.

Conclusion: FAPE-IR offers a unified and interpretable solution for all-in-one image restoration by coupling semantic planning with frequency-based restoration, demonstrating superior performance and generalization capabilities.

Abstract: All-in-One Image Restoration (AIO-IR) aims to develop a unified model that can handle multiple degradations under complex conditions. However, existing methods often rely on task-specific designs or latent routing strategies, making it hard to adapt to real-world scenarios with various degradations. We propose FAPE-IR, a Frequency-Aware Planning and Execution framework for image restoration. It uses a frozen Multimodal Large Language Model (MLLM) as a planner to analyze degraded images and generate concise, frequency-aware restoration plans. These plans guide a LoRA-based Mixture-of-Experts (LoRA-MoE) module within a diffusion-based executor, which dynamically selects high- or low-frequency experts, complemented by frequency features of the input image. To further improve restoration quality and reduce artifacts, we introduce adversarial training and a frequency regularization loss. By coupling semantic planning with frequency-based restoration, FAPE-IR offers a unified and interpretable solution for all-in-one image restoration. Extensive experiments show that FAPE-IR achieves state-of-the-art performance across seven restoration tasks and exhibits strong zero-shot generalization under mixed degradations.

Method: RIVER decouples reasoning from generation using digital twin representations of video content, employs a large language model for multi-hop reasoning to determine modifications, and uses structured instructions to guide a diffusion-based editor for pixel-level changes. Training uses reinforcement learning with rewards for reasoning accuracy and generation quality.

Result: RIVER demonstrates best performance on the proposed RVEBenchmark (100 videos with 519 implicit queries) and achieves state-of-the-art performance on two additional benchmarks (VegGIE and FiVE), surpassing six baseline methods.

Conclusion: RIVER successfully addresses the challenge of reasoning video editing by interpreting implicit queries through multi-hop reasoning, enabling more intuitive and conceptual video editing without requiring explicit spatial and temporal specifications.

Abstract: Text-driven video editing enables users to modify video content only using text queries. While existing methods can modify video content if explicit descriptions of editing targets with precise spatial locations and temporal boundaries are provided, these requirements become impractical when users attempt to conceptualize edits through implicit queries referencing semantic properties or object relationships. We introduce reasoning video editing, a task where video editing models must interpret implicit queries through multi-hop reasoning to infer editing targets before executing modifications, and a first model attempting to solve this complex task, RIVER (Reasoning-based Implicit Video Editor). RIVER decouples reasoning from generation through digital twin representations of video content that preserve spatial relationships, temporal trajectories, and semantic attributes. A large language model then processes this representation jointly with the implicit query, performing multi-hop reasoning to determine modifications, then outputs structured instructions that guide a diffusion-based editor to execute pixel-level changes. RIVER training uses reinforcement learning with rewards that evaluate reasoning accuracy and generation quality. Finally, we introduce RVEBenchmark, a benchmark of 100 videos with 519 implicit queries spanning three levels and categories of reasoning complexity specifically for reasoning video editing. RIVER demonstrates best performance on the proposed RVEBenchmark and also achieves state-of-the-art performance on two additional video editing benchmarks (VegGIE and FiVE), where it surpasses six baseline methods.

Method: Proposed RTS-Mono with lightweight encoder-decoder architecture using Lite-Encoder and multi-scale sparse fusion framework to minimize redundancy while maintaining performance.

Result: Achieved SoTA performance on KITTI dataset with 3M parameters, improving Abs Rel and Sq Rel by 5.6% and 9.8% at low resolution, and Sq Rel and RMSE by 6.1% and 1.9% at high resolution. Real-time inference at 49 FPS on Nvidia Jetson Orin.

Conclusion: RTS-Mono successfully addresses the trade-off between model efficiency and performance, enabling practical real-world deployment of self-supervised monocular depth estimation for autonomous systems.

Abstract: Depth information is crucial for autonomous driving and intelligent robot navigation. The simplicity and flexibility of self-supervised monocular depth estimation are conducive to its role in these fields. However, most existing monocular depth estimation models consume many computing resources. Although some methods have reduced the model’s size and improved computing efficiency, the performance deteriorates, seriously hindering the real-world deployment of self-supervised monocular depth estimation models in the real world. To address this problem, we proposed a real-time self-supervised monocular depth estimation method and implemented it in the real world. It is called RTS-Mono, which is a lightweight and efficient encoder-decoder architecture. The encoder is based on Lite-Encoder, and the decoder is designed with a multi-scale sparse fusion framework to minimize redundancy, ensure performance, and improve inference speed. RTS-Mono achieved state-of-the-art (SoTA) performance in high and low resolutions with extremely low parameter counts (3 M) in experiments based on the KITTI dataset. Compared with lightweight methods, RTS-Mono improved Abs Rel and Sq Rel by 5.6% and 9.8% at low resolution and improved Sq Rel and RMSE by 6.1% and 1.9% at high resolution. In real-world deployment experiments, RTS-Mono has extremely high accuracy and can perform real-time inference on Nvidia Jetson Orin at a speed of 49 FPS. Source code is available at https://github.com/ZYCheng777/RTS-Mono.

Method: Uses row-column normalization averaging with separate marginal calibration for asymmetric matching, and incorporates geometric constraints through learnable coordinate embeddings that compute compatibility scores fused with feature similarities.

Result: Experimental results on MSLS, NordLand, and Pittsburgh datasets demonstrate superior performance, validating improved matching accuracy and robustness.

Conclusion: The proposed asymmetric aggregation with geometric constraints effectively handles distributional discrepancies and enhances spatial awareness in visual place recognition.

Abstract: Visual Place Recognition (VPR) aims to match query images against a database using visual cues. State-of-the-art methods aggregate features from deep backbones to form global descriptors. Optimal transport-based aggregation methods reformulate feature-to-cluster assignment as a transport problem, but the standard Sinkhorn algorithm symmetrically treats source and target marginals, limiting effectiveness when image features and cluster centers exhibit substantially different distributions. We propose an asymmetric aggregation VPR method with geometric constraints for locally aggregated descriptors, called $A^2$GC-VPR. Our method employs row-column normalization averaging with separate marginal calibration, enabling asymmetric matching that adapts to distributional discrepancies in visual place recognition. Geometric constraints are incorporated through learnable coordinate embeddings, computing compatibility scores fused with feature similarities, thereby promoting spatially proximal features to the same cluster and enhancing spatial awareness. Experimental results on MSLS, NordLand, and Pittsburgh datasets demonstrate superior performance, validating the effectiveness of our approach in improving matching accuracy and robustness.

Method: Proposed Cascaded-ViT (CViT) with novel Cascaded-Chunk Feed Forward Network (CCFFN) that splits input features to improve parameter and FLOP efficiency.

Result: CViT-XL achieves 75.5% Top-1 accuracy on ImageNet-1K while reducing FLOPs by 15% and energy consumption by 3.3% compared to EfficientViT-M5. CViT models show lowest energy consumption across various sizes and top-ranking efficiency in Accuracy-Per-FLOP metric.

Conclusion: CViT family is suitable for deployment on battery-constrained devices due to consistently low energy consumption and high compute efficiency.

Abstract: Vision Transformers (ViTs) have demonstrated remarkable performance across a range of computer vision tasks; however, their high computational, memory, and energy demands hinder deployment on resource-constrained platforms. In this paper, we propose \emph{Cascaded-ViT (CViT)}, a lightweight and compute-efficient vision transformer architecture featuring a novel feedforward network design called \emph{Cascaded-Chunk Feed Forward Network (CCFFN)}. By splitting input features, CCFFN improves parameter and FLOP efficiency without sacrificing accuracy. Experiments on ImageNet-1K show that our \emph{CViT-XL} model achieves 75.5% Top-1 accuracy while reducing FLOPs by 15% and energy consumption by 3.3% compared to EfficientViT-M5. Across various model sizes, the CViT family consistently exhibits the lowest energy consumption, making it suitable for deployment on battery-constrained devices such as mobile phones and drones. Furthermore, when evaluated using a new metric called \emph{Accuracy-Per-FLOP (APF)}, which quantifies compute efficiency relative to accuracy, CViT models consistently achieve top-ranking efficiency. Particularly, CViT-L is 2.2% more accurate than EfficientViT-M2 while having comparable APF scores.

Method: Coffee regularizes adaptation by preventing alignment between user prompt embeddings and undesired concept embeddings, using only textual descriptions without additional training.

Result: Experimental results show Coffee effectively prevents models from learning specified undesired concepts during fine-tuning and outperforms existing methods.

Conclusion: Coffee provides a flexible, training-free approach for controllable fine-tuning of text-to-image models by using language to specify and prevent learning of undesired concepts.

Abstract: Text-to-image diffusion models can generate diverse content with flexible prompts, which makes them well-suited for customization through fine-tuning with a small amount of user-provided data. However, controllable fine-tuning that prevents models from learning undesired concepts present in the fine-tuning data, and from entangling those concepts with user prompts, remains an open challenge. It is crucial for downstream tasks like bias mitigation, preventing malicious adaptation, attribute disentanglement, and generalizable fine-tuning of diffusion policy. We propose Coffee that allows using language to specify undesired concepts to regularize the adaptation process. The crux of our method lies in keeping the embeddings of the user prompt from aligning with undesired concepts. Crucially, Coffee requires no additional training and enables flexible modification of undesired concepts by modifying textual descriptions. We evaluate Coffee by fine-tuning on images associated with user prompts paired with undesired concepts. Experimental results demonstrate that Coffee can prevent text-to-image models from learning specified undesired concepts during fine-tuning and outperforms existing methods. Code will be released upon acceptance.

Method: A 4-stage framework: (1) event-triggered multi-view video acquisition, (2) pedestrian behavior phase segmentation using TG-VLM, (3) phase-specific multi-view reasoning using PhaVR-VLM, and (4) hierarchical synthesis with LLM-generated diagnostic reports.

Result: TG-VLM achieved mIoU of 0.4881 for phase segmentation; PhaVR-VLM achieved captioning score of 33.063 and up to 64.70% QA accuracy; framework successfully translates multi-view video into actionable insights on Woven Traffic Safety dataset.

Conclusion: MP-PVIR effectively advances AI-driven traffic safety analytics by systematically processing multi-view incidents into structured diagnostic reports with prevention strategies for vehicle-infrastructure cooperative systems.

Abstract: Pedestrian-vehicle incidents remain a critical urban safety challenge, with pedestrians accounting for over 20% of global traffic fatalities. Although existing video-based systems can detect when incidents occur, they provide little insight into how these events unfold across the distinct cognitive phases of pedestrian behavior. Recent vision-language models (VLMs) have shown strong potential for video understanding, but they remain limited in that they typically process videos in isolation, without explicit temporal structuring or multi-view integration. This paper introduces Multi-view Phase-aware Pedestrian-Vehicle Incident Reasoning (MP-PVIR), a unified framework that systematically processes multi-view video streams into structured diagnostic reports through four stages: (1) event-triggered multi-view video acquisition, (2) pedestrian behavior phase segmentation, (3) phase-specific multi-view reasoning, and (4) hierarchical synthesis and diagnostic reasoning. The framework operationalizes behavioral theory by automatically segmenting incidents into cognitive phases, performing synchronized multi-view analysis within each phase, and synthesizing results into causal chains with targeted prevention strategies. Particularly, two specialized VLMs underpin the MP-PVIR pipeline: TG-VLM for behavioral phase segmentation (mIoU = 0.4881) and PhaVR-VLM for phase-aware multi-view analysis, achieving a captioning score of 33.063 and up to 64.70% accuracy on question answering. Finally, a designated large language model is used to generate comprehensive reports detailing scene understanding, behavior interpretation, causal reasoning, and prevention recommendations. Evaluation on the Woven Traffic Safety dataset shows that MP-PVIR effectively translates multi-view video data into actionable insights, advancing AI-driven traffic safety analytics for vehicle-infrastructure cooperative systems.

Method: Introduce Convolutional Nearest Neighbors (ConvNN) framework that serves as drop-in replacement for both convolution and attention layers, enabling interpolation between spatial-proximity and feature-similarity based neighbor selection.

Result: ConvNN improves accuracy on CIFAR-10/100 with hybrid VGG and outperforms standard attention in ViT. Interpolating along the spectrum provides regularization benefits by balancing local and global receptive fields.

Conclusion: Convolution and attention exist on a continuous spectrum and can be unified, providing a principled framework for designing more interpretable vision architectures.

Abstract: The shift from Convolutional Neural Networks to Transformers has reshaped computer vision, yet these two architectural families are typically viewed as fundamentally distinct. We argue that convolution and self-attention, despite their apparent differences, can be unified within a single k-nearest neighbor aggregation framework. The critical insight is that both operations are special cases of neighbor selection and aggregation; convolution selects neighbors by spatial proximity, while attention selects by feature similarity, revealing they exist on a continuous spectrum. We introduce Convolutional Nearest Neighbors (ConvNN), a unified framework that formalizes this connection. Crucially, ConvNN serves as a drop-in replacement for convolutional and attention layers, enabling systematic exploration of the intermediate spectrum between these two extremes. We validate the framework’s coherence on CIFAR-10 and CIFAR-100 classification tasks across two complementary architectures: (1) Hybrid branching in VGG improves accuracy on both CIFAR datasets by combining spatial-proximity and feature-similarity selection; and (2) ConvNN in ViT outperforms standard attention and other attention variants on both datasets. Extensive ablations on $k$ values and architectural variants reveal that interpolating along this spectrum provides regularization benefits by balancing local and global receptive fields. Our work provides a unifying framework that dissolves the apparent distinction between convolution and attention, with implications for designing more principled and interpretable vision architectures.

Method: SMART integrates audio and visual features with shot-aware token compression that selectively retains high-information tokens within each shot to reduce redundancy while preserving temporal details. It also uses refined prompt design to better utilize audio-visual cues.

Result: SMART achieves significant improvements over state-of-the-art methods, with 1.61% increase in R1@0.5 and 2.59% gain in R1@0.7 on Charades-STA, and strong performance on QVHighlights.

Conclusion: Integrating audio cues and leveraging shot-level temporal structure with selective token compression effectively improves video moment retrieval performance by providing fine-grained multimodal understanding.

Abstract: Video Moment Retrieval is a task in video understanding that aims to localize a specific temporal segment in an untrimmed video based on a natural language query. Despite recent progress in moment retrieval from videos using both traditional techniques and Multimodal Large Language Models (MLLM), most existing methods still rely on coarse temporal understanding and a single visual modality, limiting performance on complex videos. To address this, we introduce \textit{S}hot-aware \textit{M}ultimodal \textit{A}udio-enhanced \textit{R}etrieval of \textit{T}emporal \textit{S}egments (SMART), an MLLM-based framework that integrates audio cues and leverages shot-level temporal structure. SMART enriches multimodal representations by combining audio and visual features while applying \textbf{Shot-aware Token Compression}, which selectively retains high-information tokens within each shot to reduce redundancy and preserve fine-grained temporal details. We also refine prompt design to better utilize audio-visual cues. Evaluations on Charades-STA and QVHighlights show that SMART achieves significant improvements over state-of-the-art methods, including a 1.61% increase in R1@0.5 and 2.59% gain in R1@0.7 on Charades-STA.

Method: Created a large-scale dataset of image-sketch-instruction triplets for pretraining and instruction tuning, and developed O3SLM - an LVLM trained on this dataset. Evaluated on multiple sketch tasks using existing datasets (QuickDraw!, Sketchy, Tu Berlin) and their generated SketchVCL dataset.

Result: O3SLM achieves state-of-the-art performance across multiple sketch-based tasks including object localization, counting, image retrieval (SBIR and fine-grained SBIR), and visual question answering (VQA), substantially outperforming existing LVLMs in sketch comprehension and reasoning.

Conclusion: The proposed dataset and O3SLM model effectively address the sketch comprehension limitations of current LVLMs, demonstrating superior performance on diverse sketch-based reasoning tasks.

Abstract: While Large Vision Language Models (LVLMs) are increasingly deployed in real-world applications, their ability to interpret abstract visual inputs remains limited. Specifically, they struggle to comprehend hand-drawn sketches, a modality that offers an intuitive means of expressing concepts that are difficult to describe textually. We identify the primary bottleneck as the absence of a large-scale dataset that jointly models sketches, photorealistic images, and corresponding natural language instructions. To address this, we present two key contributions: (1) a new, large-scale dataset of image-sketch-instruction triplets designed to facilitate both pretraining and instruction tuning, and (2) O3SLM, an LVLM trained on this dataset. Comprehensive evaluations on multiple sketch-based tasks: (a) object localization, (b) counting, (c) image retrieval i.e., (SBIR and fine-grained SBIR), and (d) visual question answering (VQA); while incorporating the three existing sketch datasets, namely QuickDraw!, Sketchy, and Tu Berlin, along with our generated SketchVCL dataset, show that O3SLM achieves state-of-the-art performance, substantially outperforming existing LVLMs in sketch comprehension and reasoning.

Method: Two-stage feed-forward framework: 1) Coarse 6DoF pose regression using Gaussian Scene Prior-based Pose Regression Network with spatial uniform sampling and guided attention, 2) Refinement through feature matching and multi-model fusion with cross-correlation module and Weighted Multiview Predictor.

Result: Significantly reduces median rotation errors to 0.2° while achieving 2.87 FPS tracking on mobile robots - 10x speedup compared to optimization-based approaches on NeRF Synthetic, Mip-NeRF 360, and T&T+DB datasets.

Conclusion: iGaussian enables real-time camera pose estimation through direct 3D Gaussian inversion without differentiable rendering, making it suitable for robotics applications requiring fast and accurate pose estimation.

Abstract: Recent trends in SLAM and visual navigation have embraced 3D Gaussians as the preferred scene representation, highlighting the importance of estimating camera poses from a single image using a pre-built Gaussian model. However, existing approaches typically rely on an iterative \textit{render-compare-refine} loop, where candidate views are first rendered using NeRF or Gaussian Splatting, then compared against the target image, and finally, discrepancies are used to update the pose. This multi-round process incurs significant computational overhead, hindering real-time performance in robotics. In this paper, we propose iGaussian, a two-stage feed-forward framework that achieves real-time camera pose estimation through direct 3D Gaussian inversion. Our method first regresses a coarse 6DoF pose using a Gaussian Scene Prior-based Pose Regression Network with spatial uniform sampling and guided attention mechanisms, then refines it through feature matching and multi-model fusion. The key contribution lies in our cross-correlation module that aligns image embeddings with 3D Gaussian attributes without differentiable rendering, coupled with a Weighted Multiview Predictor that fuses features from Multiple strategically sampled viewpoints. Experimental results on the NeRF Synthetic, Mip-NeRF 360, and T&T+DB datasets demonstrate a significant performance improvement over previous methods, reducing median rotation errors to 0.2° while achieving 2.87 FPS tracking on mobile robots, which is an impressive 10 times speedup compared to optimization-based approaches. Code: https://github.com/pythongod-exe/iGaussian

Method: Three-stage physics-aware pipeline: proposes candidate geometric surfaces, uses transformer-based shape completion model designed for mmWave signals, and performs entropy-guided surface selection. Trained on synthetic point-clouds.

Result: Significantly outperforms state-of-the-art baselines, raising recall from 54% to 72% while maintaining high precision of 85%. Shows impressive generalization from synthetic training to real-world data.

Conclusion: Wave-Former successfully bridges wireless signals with vision-based shape completion, enabling accurate 3D reconstruction of occluded objects using mmWave technology.

Abstract: We present Wave-Former, a novel method capable of high-accuracy 3D shape reconstruction for completely occluded, diverse, everyday objects. This capability can open new applications spanning robotics, augmented reality, and logistics. Our approach leverages millimeter-wave (mmWave) wireless signals, which can penetrate common occlusions and reflect off hidden objects. In contrast to past mmWave reconstruction methods, which suffer from limited coverage and high noise, Wave-Former introduces a physics-aware shape completion model capable of inferring full 3D geometry. At the heart of Wave-Former’s design is a novel three-stage pipeline which bridges raw wireless signals with recent advancements in vision-based shape completion by incorporating physical properties of mmWave signals. The pipeline proposes candidate geometric surfaces, employs a transformer-based shape completion model designed specifically for mmWave signals, and finally performs entropy-guided surface selection. This enables Wave-Former to be trained using entirely synthetic point-clouds, while demonstrating impressive generalization to real-world data.In head-to-head comparisons with state-of-the-art baselines, Wave-Former raises recall from 54% to 72% while maintaining a high precision of 85%.

Method: Proposes RiSe framework with: 1) Asymmetric Invariant Risk Minimization (AsyIRM) that learns invariant spherical decision boundary in radial space while preserving domain cues in angular space; 2) Multimodal Synergy Disentanglement (MMSD) using self-supervised cross-sample mixing and disentanglement to enhance generalizable modal features.

Result: Theoretical analysis and experiments verify RiSe achieves state-of-the-art cross-domain performance in multimodal face anti-spoofing.

Conclusion: RiSe effectively addresses the two overlooked risks in multimodal FAS cross-domain generalization through representation and synergy invariance learning, providing a provable solution for robust multimodal anti-spoofing.

Abstract: Multimodal Face Anti-Spoofing (FAS) methods, which integrate multiple visual modalities, often suffer even more severe performance degradation than unimodal FAS when deployed in unseen domains. This is mainly due to two overlooked risks that affect cross-domain multimodal generalization. The first is the modal representation invariant risk, i.e., whether representations remain generalizable under domain shift. We theoretically show that the inherent class asymmetry in FAS (diverse spoofs vs. compact reals) enlarges the upper bound of generalization error, and this effect is further amplified in multimodal settings. The second is the modal synergy invariant risk, where models overfit to domain-specific inter-modal correlations. Such spurious synergy cannot generalize to unseen attacks in target domains, leading to performance drops. To solve these issues, we propose a provable framework, namely Multimodal Representation and Synergy Invariance Learning (RiSe). For representation risk, RiSe introduces Asymmetric Invariant Risk Minimization (AsyIRM), which learns an invariant spherical decision boundary in radial space to fit asymmetric distributions, while preserving domain cues in angular space. For synergy risk, RiSe employs Multimodal Synergy Disentanglement (MMSD), a self-supervised task enhancing intrinsic, generalizable modal features via cross-sample mixing and disentanglement. Theoretical analysis and experiments verify RiSe, which achieves state-of-the-art cross-domain performance.

Method: Designed MVI-Bench with three hierarchical levels (Visual Concept, Visual Attribute, Visual Relationship) across six categories, compiled 1,248 expert-annotated VQA instances, and introduced MVI-Sensitivity metric for fine-grained robustness evaluation.

Result: Empirical evaluation of 18 state-of-the-art LVLMs revealed pronounced vulnerabilities to misleading visual inputs, demonstrating significant robustness issues across tested models.

Conclusion: MVI-Bench provides actionable insights for developing more reliable LVLMs and fills an important gap in visual robustness evaluation, with the benchmark publicly available for future research.

Abstract: Evaluating the robustness of Large Vision-Language Models (LVLMs) is essential for their continued development and responsible deployment in real-world applications. However, existing robustness benchmarks typically focus on hallucination or misleading textual inputs, while largely overlooking the equally critical challenge posed by misleading visual inputs in assessing visual understanding. To fill this important gap, we introduce MVI-Bench, the first comprehensive benchmark specially designed for evaluating how Misleading Visual Inputs undermine the robustness of LVLMs. Grounded in fundamental visual primitives, the design of MVI-Bench centers on three hierarchical levels of misleading visual inputs: Visual Concept, Visual Attribute, and Visual Relationship. Using this taxonomy, we curate six representative categories and compile 1,248 expertly annotated VQA instances. To facilitate fine-grained robustness evaluation, we further introduce MVI-Sensitivity, a novel metric that characterizes LVLM robustness at a granular level. Empirical results across 18 state-of-the-art LVLMs uncover pronounced vulnerabilities to misleading visual inputs, and our in-depth analyses on MVI-Bench provide actionable insights that can guide the development of more reliable and robust LVLMs. The benchmark and codebase can be accessed at https://github.com/chenyil6/MVI-Bench.

Method: Object-level token merging strategy for adaptive token compression that aligns with human vision system by focusing on object-level representations.

Result: Achieves average 90% token reduction (using only 10% tokens) while maintaining 96% of vanilla model performance across multiple benchmarks.

Conclusion: The method demonstrates superior balance between compression ratio and performance compared to relevant works, with code availability.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated substantial value in unified text-image understanding and reasoning, primarily by converting images into sequences of patch-level tokens that align with their architectural paradigm. However, patch-level tokenization leads to a quadratic growth in image tokens, burdening MLLMs’ understanding and reasoning with enormous computation and memory. Additionally, the traditional patch-wise scanning tokenization workflow misaligns with the human vision cognition system, further leading to hallucination and computational redundancy. To address this issue, we propose an object-level token merging strategy for Adaptive Token compression, revealing the consistency with human vision system. The experiments are conducted on multiple comprehensive benchmarks, which show that our approach averagely, utilizes only 10% tokens while achieving almost 96% of the vanilla model’s performance. More extensive experimental results in comparison with relevant works demonstrate the superiority of our method in balancing compression ratio and performance. Our code will be available.

Method: Models sample construction as a Dominance Game, uses spatio-temporal dual weight localization for key motion regions, and employs entropy-driven dominance strategy for memory bank management.

Result: Achieves 81.1%/89.4% on NTU RGB+D 60 X-Sub/X-View and 71.2%/75.5% on NTU RGB+D 120 X-Sub/X-Set, surpassing SOTA by 0.1%, 2.7%, 1.1%, and 2.3% respectively.

Conclusion: DoGCLR demonstrates strong performance and robustness, particularly on challenging scenarios, through its game-theoretic approach to contrastive learning.

Abstract: Existing self-supervised contrastive learning methods for skeleton-based action recognition often process all skeleton regions uniformly, and adopt a first-in-first-out (FIFO) queue to store negative samples, which leads to motion information loss and non-optimal negative sample selection. To address these challenges, this paper proposes Dominance-Game Contrastive Learning network for skeleton-based action Recognition (DoGCLR), a self-supervised framework based on game theory. DoGCLR models the construction of positive and negative samples as a dynamic Dominance Game, where both sample types interact to reach an equilibrium that balances semantic preservation and discriminative strength. Specifically, a spatio-temporal dual weight localization mechanism identifies key motion regions and guides region-wise augmentations to enhance motion diversity while maintaining semantics. In parallel, an entropy-driven dominance strategy manages the memory bank by retaining high entropy (hard) negatives and replacing low-entropy (weak) ones, ensuring consistent exposure to informative contrastive signals. Extensive experiments are conducted on NTU RGB+D and PKU-MMD datasets. On NTU RGB+D 60 X-Sub/X-View, DoGCLR achieves 81.1%/89.4% accuracy, and on NTU RGB+D 120 X-Sub/X-Set, DoGCLR achieves 71.2%/75.5% accuracy, surpassing state-of-the-art methods by 0.1%, 2.7%, 1.1%, and 2.3%, respectively. On PKU-MMD Part I/Part II, DoGCLR performs comparably to the state-of-the-art methods and achieves a 1.9% higher accuracy on Part II, highlighting its strong robustness on more challenging scenarios.

Method: Curated a massive 3.8M-pair dataset with novel physically-plausible data, and developed a tailored training pipeline that fine-tunes a Diffusion Transformer with fine-grained mask and strength controls to learn robust restoration priors.

Result: UniSER significantly outperforms both specialist and generalist models, achieving robust, high-fidelity restoration in the wild by leveraging the common essence of soft effects as semi-transparent occlusions.

Conclusion: The synergistic approach of massive diverse dataset curation and diffusion transformer training enables a single versatile model to effectively address multiple soft effect degradations, demonstrating superior performance over specialized and general-purpose alternatives.

Abstract: Digital images are often degraded by soft effects such as lens flare, haze, shadows, and reflections, which reduce aesthetics even though the underlying pixels remain partially visible. The prevailing works address these degradations in isolation, developing highly specialized, specialist models that lack scalability and fail to exploit the shared underlying essences of these restoration problems. While specialist models are limited, recent large-scale pretrained generalist models offer powerful, text-driven image editing capabilities. while recent general-purpose systems (e.g., GPT-4o, Flux Kontext, Nano Banana) require detailed prompts and often fail to achieve robust removal on these fine-grained tasks or preserve identity of the scene. Leveraging the common essence of soft effects, i.e., semi-transparent occlusions, we introduce a foundational versatile model UniSER, capable of addressing diverse degradations caused by soft effects within a single framework. Our methodology centers on curating a massive 3.8M-pair dataset to ensure robustness and generalization, which includes novel, physically-plausible data to fill critical gaps in public benchmarks, and a tailored training pipeline that fine-tunes a Diffusion Transformer to learn robust restoration priors from this diverse data, integrating fine-grained mask and strength controls. This synergistic approach allows UniSER to significantly outperform both specialist and generalist models, achieving robust, high-fidelity restoration in the wild.

Method: UMEG-Net integrates human skeletons and sport-specific object keypoints into a unified graph, featuring an efficient spatio-temporal extraction module based on advanced GCN and multi-scale temporal shift. It also employs multimodal distillation to transfer knowledge from keypoint-based graphs to visual representations.

Result: The approach achieves robust performance with limited labeled data and significantly outperforms baseline models in few-shot settings.

Conclusion: UMEG-Net provides a scalable and effective solution for few-shot precise event spotting, addressing the challenge of limited labeled data in sports analytics.

Abstract: Precise event spotting (PES) aims to recognize fine-grained events at exact moments and has become a key component of sports analytics. This task is particularly challenging due to rapid succession, motion blur, and subtle visual differences. Consequently, most existing methods rely on domain-specific, end-to-end training with large labeled datasets and often struggle in few-shot conditions due to their dependence on pixel- or pose-based inputs alone. However, obtaining large labeled datasets is practically hard. We propose a Unified Multi-Entity Graph Network (UMEG-Net) for few-shot PES. UMEG-Net integrates human skeletons and sport-specific object keypoints into a unified graph and features an efficient spatio-temporal extraction module based on advanced GCN and multi-scale temporal shift. To further enhance performance, we employ multimodal distillation to transfer knowledge from keypoint-based graphs to visual representations. Our approach achieves robust performance with limited labeled data and significantly outperforms baseline models in few-shot settings, providing a scalable and effective solution for few-shot PES. Code is publicly available at https://github.com/LZYAndy/UMEG-Net.

Method: Uses global relational information with codebook-wise histogram relation learning to transfer semantics from pre-trained models to semantic codebook, plus a residual learning module to recover fine-grained details and minimize reconstruction error.

Result: Achieves state-of-the-art reconstruction performance and generation quality on ImageNet-1k benchmark, producing more uniformly distributed semantic latent representations.

Conclusion: GloTok’s global perspective approach with relational learning enables better image tokenization that facilitates high-quality image generation without requiring access to pre-trained models during training.

Abstract: Existing state-of-the-art image tokenization methods leverage diverse semantic features from pre-trained vision models for additional supervision, to expand the distribution of latent representations and thereby improve the quality of image reconstruction and generation. These methods employ a locally supervised approach for semantic supervision, which limits the uniformity of semantic distribution. However, VA-VAE proves that a more uniform feature distribution yields better generation performance. In this work, we introduce a Global Perspective Tokenizer (GloTok), which utilizes global relational information to model a more uniform semantic distribution of tokenized features. Specifically, a codebook-wise histogram relation learning method is proposed to transfer the semantics, which are modeled by pre-trained models on the entire dataset, to the semantic codebook. Then, we design a residual learning module that recovers the fine-grained details to minimize the reconstruction error caused by quantization. Through the above design, GloTok delivers more uniformly distributed semantic latent representations, which facilitates the training of autoregressive (AR) models for generating high-quality images without requiring direct access to pre-trained models during the training process. Experiments on the standard ImageNet-1k benchmark clearly show that our proposed method achieves state-of-the-art reconstruction performance and generation quality.

Method: Collected over 100 hours of naturalistic data from multiple production AV models, segmented into 32,727 key frames with synchronized camera images, GNSS/IMU data, vehicle trajectories, and detailed 2D annotations of traffic elements.

Result: Dataset provides comprehensive scenario attributes including driver intent, area types, lighting, weather conditions, and traffic density. An end-to-end motion planning model achieved 1.4m ADE on autonomous-driving frames.

Conclusion: This dataset establishes a sustainable foundation for AV behavior analysis and safety evaluation, with continuous weekly expansion of 10+ hours of new data.

Abstract: Most existing autonomous-driving datasets (e.g., KITTI, nuScenes, and the Waymo Perception Dataset), collected by human-driving mode or unidentified driving mode, can only serve as early training for the perception and prediction of autonomous vehicles (AVs). To evaluate the real behavioral safety of AVs controlled in the black box, we present the first end-to-end benchmark dataset collected entirely by autonomous-driving mode in the real world. This dataset contains over 100 hours of naturalistic data from multiple production autonomous-driving vehicle models in the market. We segment the original data into 32,727 key frames, each consisting of four synchronized camera images and high-precision GNSS/IMU data (0.8 cm localization accuracy). For each key frame, 20 Hz vehicle trajectories spanning the past 6 s and future 5 s are provided, along with detailed 2D annotations of surrounding vehicles, pedestrians, traffic lights, and traffic signs. These key frames have rich scenario-level attributes, including driver intent, area type (covering highways, urban roads, and residential areas), lighting (day, night, or dusk), weather (clear or rain), road surface (paved or unpaved), traffic and vulnerable road users (VRU) density, traffic lights, and traffic signs (warning, prohibition, and indication). To evaluate the safety of AVs, we employ an end-to-end motion planning model that predicts vehicle trajectories with an Average Displacement Error (ADE) of 1.4 m on autonomous-driving frames. The dataset continues to expand by over 10 hours of new data weekly, thereby providing a sustainable foundation for research on AV driving behavior analysis and safety evaluation.

Method: Proposes MCFormer with Global Correlation Module (GCM) that constructs global similarity affinity matrix across all images, and Local Correlation Module (LCM) that mines and aligns local features using dynamic memory bank to compensate for missing regions.

Result: Experiments on three benchmarks demonstrate state-of-the-art performance in maritime vessel re-identification.

Conclusion: MCFormer effectively suppresses adverse effects of outlier samples by modeling multi-scale correlations and integrating global-local features, achieving superior performance in vessel Re-ID tasks.

Abstract: Maritime vessel re-identification (Re-ID) plays a crucial role in advancing maritime monitoring and intelligent situational awareness systems. However, some existing vessel Re-ID methods are directly adapted from pedestrian-focused algorithms, making them ill-suited for mitigating the unique problems present in vessel images, particularly the greater intra-identity variations and more severe missing of local parts, which lead to the emergence of outlier samples within the same identity. To address these challenges, we propose the Multi-scale Correlation-aware Transformer Network (MCFormer), which explicitly models multi-scale correlations across the entire input set to suppress the adverse effects of outlier samples with intra-identity variations or local missing, incorporating two novel modules, the Global Correlation Module (GCM), and the Local Correlation Module (LCM). Specifically, GCM constructs a global similarity affinity matrix across all input images to model global correlations through feature aggregation based on inter-image consistency, rather than solely learning features from individual images as in most existing approaches. Simultaneously, LCM mines and aligns local features of positive samples with contextual similarity to extract local correlations by maintaining a dynamic memory bank, effectively compensating for missing or occluded regions in individual images. To further enhance feature robustness, MCFormer integrates global and local features that have been respectively correlated across multiple scales, effectively capturing latent relationships among image features. Experiments on three benchmarks demonstrate that MCFormer achieves state-of-the-art performance.

Method: Curriculum learning strategy with fractal softmax for hierarchical semantic learning, progressively learning from simple to complex components. Uses two-stage inference for accelerated processing.

Result: Achieves 11.65% Dice score improvement on validation set and 5.6% higher Dice score than baselines on test set. Fivefold acceleration in inference speed.

Conclusion: The framework significantly improves segmentation accuracy and efficiency, making it suitable for real-time clinical applications in aortic pathology diagnosis and treatment.

Abstract: The aorta, the body’s largest artery, is prone to pathologies such as dissection, aneurysm, and atherosclerosis, which often require timely intervention. Minimally invasive repairs involving branch vessels necessitate detailed 3D anatomical analysis. Existing methods often overlook hierarchical anatomical relationships while struggling with severe class imbalance inherent in vascular structures. We address these challenges with a curriculum learning strategy that leverages a novel fractal softmax for hierarchical semantic learning. Inspired by human cognition, our approach progressively learns anatomical constraints by decomposing complex structures from simple to complex components. The curriculum learning framework naturally addresses class imbalance by first establishing robust feature representations for dominant classes before tackling rare but anatomically critical structures, significantly accelerating model convergence in multi-class scenarios. Our two-stage inference strategy achieves up to fivefold acceleration, enhancing clinical practicality. On the validation set at epoch 50, our hierarchical semantic loss improves the Dice score of nnU-Net ResEnc M by 11.65%. The proposed model demonstrates a 5.6% higher Dice score than baselines on the test set. Experimental results show significant improvements in segmentation accuracy and efficiency, making the framework suitable for real-time clinical applications. The implementation code for this challenge entry is publicly available at: https://github.com/PengchengShi1220/AortaSeg24. The code for fractal softmax will be available at https://github.com/PengchengShi1220/fractal-softmax.

Method: Uses an agentic framework with multiple tool-calling capabilities, orchestrating specialized computer vision tools including object detection, keypoint localization, panoptic segmentation, OCR, and geometric analysis. Combines neural perception with symbolic execution.

Result: Achieves competitive performance on MMMU, MMBench, DocVQA, and MMLongBench benchmarks, extending monolithic vision-language models to production-grade visual intelligence.

Conclusion: Orion enables autonomous visual reasoning and marks a transition from passive visual understanding to active, tool-driven visual intelligence.

Abstract: We introduce Orion, a visual agent framework that can take in any modality and generate any modality. Using an agentic framework with multiple tool-calling capabilities, Orion is designed for visual AI tasks and achieves state-of-the-art results. Unlike traditional vision-language models that produce descriptive outputs, Orion orchestrates a suite of specialized computer vision tools, including object detection, keypoint localization, panoptic segmentation, Optical Character Recognition, and geometric analysis, to execute complex multi-step visual workflows. The system achieves competitive performance on MMMU, MMBench, DocVQA, and MMLongBench while extending monolithic vision-language models to production-grade visual intelligence. By combining neural perception with symbolic execution, Orion enables autonomous visual reasoning, marking a transition from passive visual understanding to active, tool-driven visual intelligence.

Method: Models global score distributions to estimate marginal perturbation of each image to dataset-level AP, computes teacher-student contribution gaps to select informative samples at each iteration, and is architecture-agnostic with minimal intrusion.

Result: Experiments on COCO dataset with multiple detectors show consistent accuracy improvements, strong robustness under low-quality data, and effective combination with knowledge distillation for further performance enhancement.

Conclusion: DetGain demonstrates potential as a general and complementary strategy for data-efficient object detection, offering consistent improvements across different architectures and conditions.

Abstract: High-quality data has become a primary driver of progress under scale laws, with curated datasets often outperforming much larger unfiltered ones at lower cost. Online data curation extends this idea by dynamically selecting training samples based on the model’s evolving state. While effective in classification and multimodal learning, existing online sampling strategies rarely extend to object detection because of its structural complexity and domain gaps. We introduce DetGain, an online data curation method specifically for object detection that estimates the marginal perturbation of each image to dataset-level Average Precision (AP) based on its prediction quality. By modeling global score distributions, DetGain efficiently estimates the global AP change and computes teacher-student contribution gaps to select informative samples at each iteration. The method is architecture-agnostic and minimally intrusive, enabling straightforward integration into diverse object detection architectures. Experiments on the COCO dataset with multiple representative detectors show consistent improvements in accuracy. DetGain also demonstrates strong robustness under low-quality data and can be effectively combined with knowledge distillation techniques to further enhance performance, highlighting its potential as a general and complementary strategy for data-efficient object detection.

Method: Distill bidirectional video diffusion teacher into causal autoregressive student using prior-driven distillation without external data; replace VAE with LeanVAE for efficiency.

Result: Achieves 35+ FPS on A100 GPUs with 100x speedup over iterative methods while matching/surpassing diffusion baseline quality in inpainting, deblurring, and super-resolution.

Conclusion: Demonstrates diffusion-based video reconstruction is practical for real-time, interactive streaming scenarios, making high-quality restoration viable for modern vision systems.

Abstract: Video inverse problems are fundamental to streaming, telepresence, and AR/VR, where high perceptual quality must coexist with tight latency constraints. Diffusion-based priors currently deliver state-of-the-art reconstructions, but existing approaches either adapt image diffusion models with ad hoc temporal regularizers - leading to temporal artifacts - or rely on native video diffusion models whose iterative posterior sampling is far too slow for real-time use. We introduce InstantViR, an amortized inference framework for ultra-fast video reconstruction powered by a pre-trained video diffusion prior. We distill a powerful bidirectional video diffusion model (teacher) into a causal autoregressive student that maps a degraded video directly to its restored version in a single forward pass, inheriting the teacher’s strong temporal modeling while completely removing iterative test-time optimization. The distillation is prior-driven: it only requires the teacher diffusion model and known degradation operators, and does not rely on externally paired clean/noisy video data. To further boost throughput, we replace the video-diffusion backbone VAE with a high-efficiency LeanVAE via an innovative teacher-space regularized distillation scheme, enabling low-latency latent-space processing. Across streaming random inpainting, Gaussian deblurring and super-resolution, InstantViR matches or surpasses the reconstruction quality of diffusion-based baselines while running at over 35 FPS on NVIDIA A100 GPUs, achieving up to 100 times speedups over iterative video diffusion solvers. These results show that diffusion-based video reconstruction is compatible with real-time, interactive, editable, streaming scenarios, turning high-quality video restoration into a practical component of modern vision systems.

Method: Lift single RGBD image to create initial point cloud, maintain and expand world model, then optimize using Gaussian splatting representation.

Result: Extensive experiments show strong generalization capability and superior performance in generating world models and synthesizing high-fidelity, consistent novel views across diverse datasets.

Conclusion: Gen3d effectively generates high-quality, wide-scope 3D scenes from single images, advancing 3D scene generation for AI applications.

Abstract: Despite recent advancements in neural 3D reconstruction, the dependence on dense multi-view captures restricts their broader applicability. Additionally, 3D scene generation is vital for advancing embodied AI and world models, which depend on diverse, high-quality scenes for learning and evaluation. In this work, we propose Gen3d, a novel method for generation of high-quality, wide-scope, and generic 3D scenes from a single image. After the initial point cloud is created by lifting the RGBD image, Gen3d maintains and expands its world model. The 3D scene is finalized through optimizing a Gaussian splatting representation. Extensive experiments on diverse datasets demonstrate the strong generalization capability and superior performance of our method in generating a world model and Synthesizing high-fidelity and consistent novel views.

Method: Formulates BFR as measurement-constrained generative task using controlled degradations of coarse restorations, enabling posterior-guided sampling in text-to-image diffusion with Forward and Reverse Measurement constraints.

Result: MCS generates prompt-aligned results and outperforms existing BFR methods in experiments.

Conclusion: The proposed MCS approach effectively enables diverse face reconstructions conditioned on textual prompts while maintaining alignment with input structures.

Abstract: Blind face restoration (BFR) may correspond to multiple plausible high-quality (HQ) reconstructions under extremely low-quality (LQ) inputs. However, existing methods typically produce deterministic results, struggling to capture this one-to-many nature. In this paper, we propose a Measurement-Constrained Sampling (MCS) approach that enables diverse LQ face reconstructions conditioned on different textual prompts. Specifically, we formulate BFR as a measurement-constrained generative task by constructing an inverse problem through controlled degradations of coarse restorations, which allows posterior-guided sampling within text-to-image diffusion. Measurement constraints include both Forward Measurement, which ensures results align with input structures, and Reverse Measurement, which produces projection spaces, ensuring that the solution can align with various prompts. Experiments show that our MCS can generate prompt-aligned results and outperforms existing BFR methods. Codes will be released after acceptance.

Method: Uses a high-capacity segmentation foundation model to guide lightweight clients through dual knowledge distillation and adaptive agreement mechanism for pixel-level supervision refinement.

Result: Experiments on skin lesion and polyp segmentation show SAM-Fed consistently outperforms state-of-the-art FSSL methods in both homogeneous and heterogeneous settings.

Conclusion: SAM-Fed effectively addresses federated semi-supervised learning challenges by leveraging foundation models to improve pseudo-label quality while accommodating client device constraints.

Abstract: Medical image segmentation is clinically important, yet data privacy and the cost of expert annotation limit the availability of labeled data. Federated semi-supervised learning (FSSL) offers a solution but faces two challenges: pseudo-label reliability depends on the strength of local models, and client devices often require compact or heterogeneous architectures due to limited computational resources. These constraints reduce the quality and stability of pseudo-labels, while large models, though more accurate, cannot be trained or used for routine inference on client devices. We propose SAM-Fed, a federated semi-supervised framework that leverages a high-capacity segmentation foundation model to guide lightweight clients during training. SAM-Fed combines dual knowledge distillation with an adaptive agreement mechanism to refine pixel-level supervision. Experiments on skin lesion and polyp segmentation across homogeneous and heterogeneous settings show that SAM-Fed consistently outperforms state-of-the-art FSSL methods.

Method: Uses an autoregressive diffusion model that processes input audio in streaming manner, selecting limited past frames as historical motion context combined with audio input to create dynamic conditions for iterative facial motion generation.

Result: Achieves flexibility with varying audio lengths and low latency independent of audio duration, enabling real-time synthesis with high-quality results.

Conclusion: The proposed streaming approach effectively addresses latency and length limitations of previous methods while maintaining high-quality facial animation.

Abstract: This paper focuses on the task of speech-driven 3D facial animation, which aims to generate realistic and synchronized facial motions driven by speech inputs.Recent methods have employed audio-conditioned diffusion models for 3D facial animation, achieving impressive results in generating expressive and natural animations.However, these methods process the whole audio sequences in a single pass, which poses two major challenges: they tend to perform poorly when handling audio sequences that exceed the training horizon and will suffer from significant latency when processing long audio inputs. To address these limitations, we propose a novel autoregressive diffusion model that processes input audio in a streaming manner. This design ensures flexibility with varying audio lengths and achieves low latency independent of audio duration. Specifically, we select a limited number of past frames as historical motion context and combine them with the audio input to create a dynamic condition. This condition guides the diffusion process to iteratively generate facial motion frames, enabling real-time synthesis with high-quality results. Additionally, we implemented a real-time interactive demo, highlighting the effectiveness and efficiency of our approach. We will release the code at https://zju3dv.github.io/StreamingTalker/.

Method: Proposed LSP-YOLO based on YOLOv11-Pose, integrating partial convolution (PConv) and Similarity-Aware Activation Module (SimAM) in Light-C3k2 module. Uses pointwise convolution to directly map keypoints to posture classes with intermediate supervision for efficient pose-classification fusion.

Result: LSP-YOLO-n achieved 94.2% accuracy and 251 FPS on PC with 1.9MB model size. Demonstrated real-time high-accuracy inference on SV830C + GC030A platform. Dataset of 5,000 images across 6 posture categories was constructed.

Conclusion: The approach is highly efficient, lightweight, and deployable, suitable for smart classrooms, rehabilitation, and human-computer interaction applications.

Abstract: With the rise in sedentary behavior, health problems caused by poor sitting posture have drawn increasing attention. Most existing methods, whether using invasive sensors or computer vision, rely on two-stage pipelines, which result in high intrusiveness, intensive computation, and poor real-time performance on embedded edge devices. Inspired by YOLOv11-Pose, a lightweight single-stage network for sitting posture recognition on embedded edge devices termed LSP-YOLO was proposed. By integrating partial convolution(PConv) and Similarity-Aware Activation Module(SimAM), a lightweight module, Light-C3k2, was designed to reduce computational cost while maintaining feature extraction capability. In the recognition head, keypoints were directly mapped to posture classes through pointwise convolution, and intermediate supervision was employed to enable efficient fusion of pose estimation and classification. Furthermore, a dataset containing 5,000 images across six posture categories was constructed for model training and testing. The smallest trained model, LSP-YOLO-n, achieved 94.2% accuracy and 251 Fps on personal computer(PC) with a model size of only 1.9 MB. Meanwhile, real-time and high-accuracy inference under constrained computational resources was demonstrated on the SV830C + GC030A platform. The proposed approach is characterized by high efficiency, lightweight design and deployability, making it suitable for smart classrooms, rehabilitation, and human-computer interaction applications.

Method: Uses quotient space representations to decouple motion geometry from coordinate redundancy, with data perception module for geometric dimension reduction and network perception module with restorative learning through masking/noise injection for auxiliary supervision.

Result: Achieves state-of-the-art performance with 16.3% improvement on H3.6M, 13.9% on CMU Mocap, and 10.1% on 3DPW datasets.

Conclusion: APS provides an effective active perceptual strategy that is model-agnostic and significantly enhances motion prediction performance through explicit motion encoding and auxiliary learning objectives.

Abstract: Forecasting 3D human motion is an important embodiment of fine-grained understanding and cognition of human behavior by artificial agents. Current approaches excessively rely on implicit network modeling of spatiotemporal relationships and motion characteristics, falling into the passive learning trap that results in redundant and monotonous 3D coordinate information acquisition while lacking actively guided explicit learning mechanisms. To overcome these issues, we propose an Active Perceptual Strategy (APS) for human motion prediction, leveraging quotient space representations to explicitly encode motion properties while introducing auxiliary learning objectives to strengthen spatio-temporal modeling. Specifically, we first design a data perception module that projects poses into the quotient space, decoupling motion geometry from coordinate redundancy. By jointly encoding tangent vectors and Grassmann projections, this module simultaneously achieves geometric dimension reduction, semantic decoupling, and dynamic constraint enforcement for effective motion pose characterization. Furthermore, we introduce a network perception module that actively learns spatio-temporal dependencies through restorative learning. This module deliberately masks specific joints or injects noise to construct auxiliary supervision signals. A dedicated auxiliary learning network is designed to actively adapt and learn from perturbed information. Notably, APS is model agnostic and can be integrated with different prediction models to enhance active perceptual. The experimental results demonstrate that our method achieves the new state-of-the-art, outperforming existing methods by large margins: 16.3% on H3.6M, 13.9% on CMU Mocap, and 10.1% on 3DPW.

Method: Uses dense self-training with semantically-aware hierarchical normalization, weak supervision via pairwise ordinal depth annotations, and weight regularization to anchor LoRA updates while preserving generalizable knowledge.

Result: Extensive experiments show WeSTAR consistently improves generalization and achieves state-of-the-art performance across diverse benchmarks, including realistic and corrupted out-of-distribution datasets.

Conclusion: WeSTAR effectively enhances the robustness of monocular depth estimation foundation models through parameter-efficient adaptation with multiple regularization techniques, demonstrating superior performance in challenging scenarios.

Abstract: The emergence of foundation models has substantially advanced zero-shot generalization in monocular depth estimation (MDE), as exemplified by the Depth Anything series. However, given access to some data from downstream tasks, a natural question arises: can the performance of these models be further improved? To this end, we propose WeSTAR, a parameter-efficient framework that performs Weakly supervised Self-Training Adaptation with Regularization, designed to enhance the robustness of MDE foundation models in unseen and diverse domains. We first adopt a dense self-training objective as the primary source of structural self-supervision. To further improve robustness, we introduce semantically-aware hierarchical normalization, which exploits instance-level segmentation maps to perform more stable and multi-scale structural normalization. Beyond dense supervision, we introduce a cost-efficient weak supervision in the form of pairwise ordinal depth annotations to further guide the adaptation process, which enforces informative ordinal constraints to mitigate local topological errors. Finally, a weight regularization loss is employed to anchor the LoRA updates, ensuring training stability and preserving the model’s generalizable knowledge. Extensive experiments on both realistic and corrupted out-of-distribution datasets under diverse and challenging scenarios demonstrate that WeSTAR consistently improves generalization and achieves state-of-the-art performance across a wide range of benchmarks.

Method: Mobile app developed using Flutter framework with Google ML Kit for on-device real-time analysis, validated against 45 manually annotated patient videos by ophthalmology specialists

Result: 98.4% precision, 96.9% recall, and 98.3% overall accuracy in blink detection

Conclusion: Bapp is a reliable, portable, and accessible tool for monitoring eyelid movements, offering alternative to manual blink counting for clinical use

Abstract: Blinking is a vital physiological process that protects and maintains the health of the ocular surface. Objective assessment of eyelid movements remains challenging due to the complexity, cost, and limited clinical applicability of existing tools. This study presents the clinical validation of Bapp (Blink Application), a mobile application developed using the Flutter framework and integrated with Google ML Kit for on-device, real-time analysis of eyelid movements. The validation occurred using 45 videos from real patients, whose blinks were manually annotated by ophthalmology specialists from the Paulista School of Medicine of the Federal University of Sao Paulo (EPM-UNIFESP) to serve as the ground truth. Bapp’s performance was evaluated using standard metrics, including Precision, Recall, and F1-Score, with results demonstrating 98.4% precision, 96.9% recall, and an overall accuracy of 98.3%. These outcomes confirm the reliability of Bapp as a portable, accessible, and objective tool for monitoring both normal and abnormal eyelid movements. The application offers a promising alternative to traditional manual blink counting, supporting continuous ocular health monitoring and postoperative evaluation in clinical environments.

Method: Uses a Pose Generator with Confidence (PGC) for pose estimation and Pose-Aware Spatio-Temporal Alignment Transformer (PASTAT) for confidence-aware spatial alignment with temporal context. Also introduces V2VLoc dataset for training.

Result: Achieves state-of-the-art performance on V2VLoc dataset under GNSS-denied conditions and shows effectiveness on real-world V2V4Real dataset.

Conclusion: The proposed framework provides robust collaborative perception without GNSS by effectively handling localization errors through confidence-aware alignment and temporal modeling.

Abstract: Multi-agents rely on accurate poses to share and align observations, enabling a collaborative perception of the environment. However, traditional GNSS-based localization often fails in GNSS-denied environments, making consistent feature alignment difficult in collaboration. To tackle this challenge, we propose a robust GNSS-free collaborative perception framework based on LiDAR localization. Specifically, we propose a lightweight Pose Generator with Confidence (PGC) to estimate compact pose and confidence representations. To alleviate the effects of localization errors, we further develop the Pose-Aware Spatio-Temporal Alignment Transformer (PASTAT), which performs confidence-aware spatial alignment while capturing essential temporal context. Additionally, we present a new simulation dataset, V2VLoc, which can be adapted for both LiDAR localization and collaborative detection tasks. V2VLoc comprises three subsets: Town1Loc, Town4Loc, and V2VDet. Town1Loc and Town4Loc offer multi-traversal sequences for training in localization tasks, whereas V2VDet is specifically intended for the collaborative detection task. Extensive experiments conducted on the V2VLoc dataset demonstrate that our approach achieves state-of-the-art performance under GNSS-denied conditions. We further conduct extended experiments on the real-world V2V4Real dataset to validate the effectiveness and generalizability of PASTAT.

Method: Uses 3D PAE with global camouflage texture, 3D stereo matching rendering module for real-world alignment, and novel merging attack with fine-grained optimization for seamless background integration.

Result: PAEs successfully fool stereo models into producing erroneous depth information with enhanced stealth and lethality compared to existing hiding attacks.

Conclusion: The proposed 3D texture-enabled physical adversarial attack effectively compromises stereo depth estimation in autonomous driving scenarios while maintaining visual consistency across viewpoints.

Abstract: Though deep neural models adopted to realize the perception of autonomous driving have proven vulnerable to adversarial examples, known attacks often leverage 2D patches and target mostly monocular perception. Therefore, the effectiveness of Physical Adversarial Examples (PAEs) on stereo-based binocular depth estimation remains largely unexplored. To this end, we propose the first texture-enabled physical adversarial attack against stereo matching models in the context of autonomous driving. Our method employs a 3D PAE with global camouflage texture rather than a local 2D patch-based one, ensuring both visual consistency and attack effectiveness across different viewpoints of stereo cameras. To cope with the disparity effect of these cameras, we also propose a new 3D stereo matching rendering module that allows the PAE to be aligned with real-world positions and headings in binocular vision. We further propose a novel merging attack that seamlessly blends the target into the environment through fine-grained PAE optimization. It has significantly enhanced stealth and lethality upon existing hiding attacks that fail to get seamlessly merged into the background. Extensive evaluations show that our PAEs can successfully fool the stereo models into producing erroneous depth information.

Method: Created ManipBench with 450K+ manipulated images from 25 editing models across 12 categories, with 100K annotated with bounding boxes and explanations. Proposed ManipShield using MLLM with contrastive LoRA fine-tuning and task-specific decoders.

Result: ManipShield achieves state-of-the-art performance on ManipBench and public datasets, demonstrating strong generalization to unseen manipulation models.

Conclusion: ManipBench addresses limitations of existing benchmarks, and ManipShield provides an effective unified solution for manipulation detection, localization, and explanation with strong generalization capabilities.

Abstract: With the rapid advancement of generative models, powerful image editing methods now enable diverse and highly realistic image manipulations that far surpass traditional deepfake techniques, posing new challenges for manipulation detection. Existing image manipulation detection and localization (IMDL) benchmarks suffer from limited content diversity, narrow generative-model coverage, and insufficient interpretability, which hinders the generalization and explanation capabilities of current manipulation detection methods. To address these limitations, we introduce \textbf{ManipBench}, a large-scale benchmark for image manipulation detection and localization focusing on AI-edited images. ManipBench contains over 450K manipulated images produced by 25 state-of-the-art image editing models across 12 manipulation categories, among which 100K images are further annotated with bounding boxes, judgment cues, and textual explanations to support interpretable detection. Building upon ManipBench, we propose \textbf{ManipShield}, an all-in-one model based on a Multimodal Large Language Model (MLLM) that leverages contrastive LoRA fine-tuning and task-specific decoders to achieve unified image manipulation detection, localization, and explanation. Extensive experiments on ManipBench and several public datasets demonstrate that ManipShield achieves state-of-the-art performance and exhibits strong generality to unseen manipulation models. Both ManipBench and ManipShield will be released upon publication.

Method: Uses tailored 2D Gaussian splatting to generate latent tensor and 1D Gaussian splatting to generate transform matrix, creating a continuous representation framework. Developed unsupervised GSLR-based multi-dimensional image recovery model for evaluation.

Result: Extensive experiments show GSLR consistently outperforms state-of-the-art methods in multi-dimensional image recovery, particularly in capturing local high-frequency information.

Conclusion: GSLR framework provides powerful representation capability for multi-dimensional images through complementary 2D and 1D Gaussian splatting, effectively addressing limitations of traditional t-SVD methods.

Abstract: Tensor singular value decomposition (t-SVD) is a promising tool for multi-dimensional image representation, which decomposes a multi-dimensional image into a latent tensor and an accompanying transform matrix. However, two critical limitations of t-SVD methods persist: (1) the approximation of the latent tensor (e.g., tensor factorizations) is coarse and fails to accurately capture spatial local high-frequency information; (2) The transform matrix is composed of fixed basis atoms (e.g., complex exponential atoms in DFT and cosine atoms in DCT) and cannot precisely capture local high-frequency information along the mode-3 fibers. To address these two limitations, we propose a Gaussian Splatting-based Low-rank tensor Representation (GSLR) framework, which compactly and continuously represents multi-dimensional images. Specifically, we leverage tailored 2D Gaussian splatting and 1D Gaussian splatting to generate the latent tensor and transform matrix, respectively. The 2D and 1D Gaussian splatting are indispensable and complementary under this representation framework, which enjoys a powerful representation capability, especially for local high-frequency information. To evaluate the representation ability of the proposed GSLR, we develop an unsupervised GSLR-based multi-dimensional image recovery model. Extensive experiments on multi-dimensional image recovery demonstrate that GSLR consistently outperforms state-of-the-art methods, particularly in capturing local high-frequency information.

Method: Proposes a dual-query critic signal derived from VLM’s Yes/No log-odds that assesses semantic fidelity and geometric coherence. Applies this as reward objective for optimization-based pipelines and as test-time guidance for feed-forward pipelines.

Result: Significantly outperforms existing methods on standard benchmarks for optimization-based pipelines, and actively corrects severe spatial errors in feed-forward pipelines by steering iterative sampling processes.

Conclusion: VLM3D establishes a principled and generalizable framework to inject VLM’s language-grounded understanding of semantics and space into diverse 3D generative pipelines.

Abstract: Text-to-3D generation has advanced rapidly, yet state-of-the-art models, encompassing both optimization-based and feed-forward architectures, still face two fundamental limitations. First, they struggle with coarse semantic alignment, often failing to capture fine-grained prompt details. Second, they lack robust 3D spatial understanding, leading to geometric inconsistencies and catastrophic failures in part assembly and spatial relationships. To address these challenges, we propose VLM3D, a general framework that repurposes large vision-language models (VLMs) as powerful, differentiable semantic and spatial critics. Our core contribution is a dual-query critic signal derived from the VLM’s Yes or No log-odds, which assesses both semantic fidelity and geometric coherence. We demonstrate the generality of this guidance signal across two distinct paradigms: (1) As a reward objective for optimization-based pipelines, VLM3D significantly outperforms existing methods on standard benchmarks. (2) As a test-time guidance module for feed-forward pipelines, it actively steers the iterative sampling process of SOTA native 3D models to correct severe spatial errors. VLM3D establishes a principled and generalizable path to inject the VLM’s rich, language-grounded understanding of both semantics and space into diverse 3D generative pipelines.

Method: Construct IDP with source feature embeddings as codebook, reconstruct target domain features using this codebook, and develop fast domain alignment using proxies as guidance for target feature transformation.

Result: The proposed model surpasses state-of-the-art models by a significant margin on 8 cross-domain few-shot learning benchmarks.

Conclusion: Intermediate domain proxies with collaborative learning of reconstruction and feature transformation effectively address CDFSL challenges and achieve superior performance.

Abstract: Cross-Domain Few-Shot Learning (CDFSL) endeavors to transfer generalized knowledge from the source domain to target domains using only a minimal amount of training data, which faces a triplet of learning challenges in the meantime, i.e., semantic disjoint, large domain discrepancy, and data scarcity. Different from predominant CDFSL works focused on generalized representations, we make novel attempts to construct Intermediate Domain Proxies (IDP) with source feature embeddings as the codebook and reconstruct the target domain feature with this learned codebook. We then conduct an empirical study to explore the intrinsic attributes from perspectives of visual styles and semantic contents in intermediate domain proxies. Reaping benefits from these attributes of intermediate domains, we develop a fast domain alignment method to use these proxies as learning guidance for target domain feature transformation. With the collaborative learning of intermediate domain reconstruction and target feature transformation, our proposed model is able to surpass the state-of-the-art models by a margin on 8 cross-domain few-shot learning benchmarks.

Method: Proposed NeuralSSD with a novel energy equation balancing point cloud reliability and a 3D convolutional network that learns spatial information for superior optimization.

Result: Achieved state-of-the-art results on ShapeNet and Matterport datasets, with highly accurate and stable surface reconstruction that closely adheres to input points.

Conclusion: NeuralSSD provides an effective solution for high-quality 3D surface reconstruction from point clouds by ensuring tight data fitting and leveraging learned inductive biases.

Abstract: We proposed a generalized method, NeuralSSD, for reconstructing a 3D implicit surface from the widely-available point cloud data. NeuralSSD is a solver-based on the neural Galerkin method, aimed at reconstructing higher-quality and accurate surfaces from input point clouds. Implicit method is preferred due to its ability to accurately represent shapes and its robustness in handling topological changes. However, existing parameterizations of implicit fields lack explicit mechanisms to ensure a tight fit between the surface and input data. To address this, we propose a novel energy equation that balances the reliability of point cloud information. Additionally, we introduce a new convolutional network that learns three-dimensional information to achieve superior optimization results. This approach ensures that the reconstructed surface closely adheres to the raw input points and infers valuable inductive biases from point clouds, resulting in a highly accurate and stable surface reconstruction. NeuralSSD is evaluated on a variety of challenging datasets, including the ShapeNet and Matterport datasets, and achieves state-of-the-art results in terms of both surface reconstruction accuracy and generalizability.

Method: Three key innovations: (1) Retrieve-Perceive-Review reasoning process for global exploration and local analysis, (2) structured video knowledge base with entity graphs and multi-granularity tools, (3) open-source model ensemble combining reasoning LLMs with CV models and VLM.

Result: Experiments on LVBench, VideoMME-Long, LongVideoBench, and Charades-STA show competitive performance while offering superior interpretability.

Conclusion: AVI provides an effective training-free framework for video understanding that achieves strong performance through human-inspired reasoning processes and open-source model integration, eliminating dependency on proprietary APIs or RL training.

Abstract: Video understanding requires not only visual recognition but also complex reasoning. While Vision-Language Models (VLMs) demonstrate impressive capabilities, they typically process videos largely in a single-pass manner with limited support for evidence revisit and iterative refinement. While recently emerging agent-based methods enable long-horizon reasoning, they either depend heavily on expensive proprietary models or require extensive agentic RL training. To overcome these limitations, we propose Agentic Video Intelligence (AVI), a flexible and training-free framework that can mirror human video comprehension through system-level design and optimization. AVI introduces three key innovations: (1) a human-inspired three-phase reasoning process (Retrieve-Perceive-Review) that ensures both sufficient global exploration and focused local analysis, (2) a structured video knowledge base organized through entity graphs, along with multi-granularity integrated tools, constituting the agent’s interaction environment, and (3) an open-source model ensemble combining reasoning LLMs with lightweight base CV models and VLM, eliminating dependence on proprietary APIs or RL training. Experiments on LVBench, VideoMME-Long, LongVideoBench, and Charades-STA demonstrate that AVI achieves competitive performance while offering superior interpretability.

Method: Two-module framework: implicit neural unsigned distance field learns preoperative bone model, and MLP-based registration module performs global initialization and local refinement using transformation hypotheses to align intraoperative point clouds.

Result: Achieved mean RRE/RTE of 1.68°/1.86 mm on UltraBones100k, 1.88°/1.89 mm on UltraBones-Hip, and 3.79°/2.45 mm on SpineDepth, matching or surpassing existing methods across multiple datasets.

Conclusion: NeuralBoneReg demonstrates strong generalizability across anatomies and modalities, providing robust and accurate cross-modal alignment for computer-assisted orthopedic surgery without requiring inter-subject training data.

Abstract: In computer- and robot-assisted orthopedic surgery (CAOS), patient-specific surgical plans derived from preoperative imaging define target locations and implant trajectories. During surgery, these plans must be accurately transferred, relying on precise cross-registration between preoperative and intraoperative data. However, substantial modality heterogeneity across imaging modalities makes this registration challenging and error-prone. Robust, automatic, and modality-agnostic bone surface registration is therefore clinically important. We propose NeuralBoneReg, a self-supervised, surface-based framework that registers bone surfaces using 3D point clouds as a modality-agnostic representation. NeuralBoneReg includes two modules: an implicit neural unsigned distance field (UDF) that learns the preoperative bone model, and an MLP-based registration module that performs global initialization and local refinement by generating transformation hypotheses to align the intraoperative point cloud with the neural UDF. Unlike SOTA supervised methods, NeuralBoneReg operates in a self-supervised manner, without requiring inter-subject training data. We evaluated NeuralBoneReg against baseline methods on two publicly available multi-modal datasets: a CT-ultrasound dataset of the fibula and tibia (UltraBones100k) and a CT-RGB-D dataset of spinal vertebrae (SpineDepth). The evaluation also includes a newly introduced CT–ultrasound dataset of cadaveric subjects containing femur and pelvis (UltraBones-Hip), which will be made publicly available. NeuralBoneReg matches or surpasses existing methods across all datasets, achieving mean RRE/RTE of 1.68°/1.86 mm on UltraBones100k, 1.88°/1.89 mm on UltraBones-Hip, and 3.79°/2.45 mm on SpineDepth. These results demonstrate strong generalizability across anatomies and modalities, providing robust and accurate cross-modal alignment for CAOS.

Method: Iterative framework using Neural Attenuation Field representation with dual-branch conditional diffusion model, employing Angle-Prior Guided Projection Synthesis and Diffusion-driven Reuse Projection Refinement.

Result: Achieves best performance under ultra-sparse-view conditions on both simulated 3D CT volumes and real projection data, progressively enhancing projection completeness and reconstruction fidelity.

Conclusion: Diff-NAF effectively addresses ultra-sparse-view CT reconstruction challenges through iterative refinement, yielding high-quality reconstructions where traditional methods fail.

Abstract: Multi-source stationary computed tomography (CT) has recently attracted attention for its ability to achieve rapid image reconstruction, making it suitable for time-sensitive clinical and industrial applications. However, practical systems are often constrained by ultra-sparse-view sampling, which significantly degrades reconstruction quality. Traditional methods struggle under ultra-sparse-view settings, where interpolation becomes inaccurate and the resulting reconstructions are unsatisfactory. To address this challenge, this study proposes Diffusion-Refined Neural Attenuation Fields (Diff-NAF), an iterative framework tailored for multi-source stationary CT under ultra-sparse-view conditions. Diff-NAF combines a Neural Attenuation Field representation with a dual-branch conditional diffusion model. The process begins by training an initial NAF using ultra-sparse-view projections. New projections are then generated through an Angle-Prior Guided Projection Synthesis strategy that exploits inter view priors, and are subsequently refined by a Diffusion-driven Reuse Projection Refinement Module. The refined projections are incorporated as pseudo-labels into the training set for the next iteration. Through iterative refinement, Diff-NAF progressively enhances projection completeness and reconstruction fidelity under ultra-sparse-view conditions, ultimately yielding high-quality CT reconstructions. Experimental results on multiple simulated 3D CT volumes and real projection data demonstrate that Diff-NAF achieves the best performance under ultra-sparse-view conditions.

Method: Proposes Dental3R pipeline with two key components: 1) Geometry-Aware Pairing Strategy (GAPS) to select high-value image pairs for stable geometry initialization and reduced memory usage, and 2) 3DGS training with wavelet-regularized objective using discrete wavelet transform to enforce band-limited fidelity and preserve fine dental details while suppressing artifacts.

Result: Validated on 950 clinical cases and 195 video-based test cases. Dental3R effectively handles sparse, unposed inputs and achieves superior novel view synthesis quality for dental occlusion visualization, outperforming state-of-the-art methods.

Conclusion: Dental3R provides a robust solution for high-fidelity 3D dental reconstruction from sparse intraoral photographs, enabling remote tele-orthodontics by overcoming challenges of conventional methods and preserving critical diagnostic details through intelligent pairing and wavelet regularization.

Abstract: Intraoral 3D reconstruction is fundamental to digital orthodontics, yet conventional methods like intraoral scanning are inaccessible for remote tele-orthodontics, which typically relies on sparse smartphone imagery. While 3D Gaussian Splatting (3DGS) shows promise for novel view synthesis, its application to the standard clinical triad of unposed anterior and bilateral buccal photographs is challenging. The large view baselines, inconsistent illumination, and specular surfaces common in intraoral settings can destabilize simultaneous pose and geometry estimation. Furthermore, sparse-view photometric supervision often induces a frequency bias, leading to over-smoothed reconstructions that lose critical diagnostic details. To address these limitations, we propose \textbf{Dental3R}, a pose-free, graph-guided pipeline for robust, high-fidelity reconstruction from sparse intraoral photographs. Our method first constructs a Geometry-Aware Pairing Strategy (GAPS) to intelligently select a compact subgraph of high-value image pairs. The GAPS focuses on correspondence matching, thereby improving the stability of the geometry initialization and reducing memory usage. Building on the recovered poses and point cloud, we train the 3DGS model with a wavelet-regularized objective. By enforcing band-limited fidelity using a discrete wavelet transform, our approach preserves fine enamel boundaries and interproximal edges while suppressing high-frequency artifacts. We validate our approach on a large-scale dataset of 950 clinical cases and an additional video-based test set of 195 cases. Experimental results demonstrate that Dental3R effectively handles sparse, unposed inputs and achieves superior novel view synthesis quality for dental occlusion visualization, outperforming state-of-the-art methods.

Method: Propose StepsNet that separates features along channel dimension and enables progressive learning through stacking blocks with increasing width, serving as a versatile macro design applicable to various models.

Result: Extensive experiments show consistent outperformance over residual models across diverse tasks including image classification, object detection, semantic segmentation, and language modeling.

Conclusion: StepsNet serves as a superior generalization of the widely adopted residual architecture, effectively bridging the gap between theoretical potential and practical performance of deep models.

Abstract: Scaling up network depth is a fundamental pursuit in neural architecture design, as theory suggests that deeper models offer exponentially greater capability. Benefiting from the residual connections, modern neural networks can scale up to more than one hundred layers and enjoy wide success. However, as networks continue to deepen, current architectures often struggle to realize their theoretical capacity improvements, calling for more advanced designs to further unleash the potential of deeper networks. In this paper, we identify two key barriers that obstruct residual models from scaling deeper: shortcut degradation and limited width. Shortcut degradation hinders deep-layer learning, while the inherent depth-width trade-off imposes limited width. To mitigate these issues, we propose a generalized residual architecture dubbed Step by Step Network (StepsNet) to bridge the gap between theoretical potential and practical performance of deep models. Specifically, we separate features along the channel dimension and let the model learn progressively via stacking blocks with increasing width. The resulting method mitigates the two identified problems and serves as a versatile macro design applicable to various models. Extensive experiments show that our method consistently outperforms residual models across diverse tasks, including image classification, object detection, semantic segmentation, and language modeling. These results position StepsNet as a superior generalization of the widely adopted residual architecture.

Method: ArchMap uses geometry-aware arch-flattening to standardize 3D meshes into aligned multi-view projections, and a Dental Knowledge Base (DKB) encoding hierarchical tooth ontology, dentition-stage policies, and clinical semantics for symbolic reasoning.

Result: Validated on 1060 orthodontic cases, ArchMap achieves robust performance in tooth counting, anatomical partitioning, dentition-stage classification, and identifying clinical conditions like crowding, missing teeth, prosthetics, and caries. Outperforms supervised pipelines and VLM baselines with higher accuracy and stability.

Conclusion: Combining geometric normalization with ontology-guided multimodal reasoning provides a practical, scalable solution for structured 3D intraoral scan analysis without training requirements, enabling robust deployment in digital orthodontics.

Abstract: A structured understanding of intraoral 3D scans is essential for digital orthodontics. However, existing deep-learning approaches rely heavily on modality-specific training, large annotated datasets, and controlled scanning conditions, which limit generalization across devices and hinder deployment in real clinical workflows. Moreover, raw intraoral meshes exhibit substantial variation in arch pose, incomplete geometry caused by occlusion or tooth contact, and a lack of texture cues, making unified semantic interpretation highly challenging. To address these limitations, we propose ArchMap, a training-free and knowledge-guided framework for robust structured dental understanding. ArchMap first introduces a geometry-aware arch-flattening module that standardizes raw 3D meshes into spatially aligned, continuity-preserving multi-view projections. We then construct a Dental Knowledge Base (DKB) encoding hierarchical tooth ontology, dentition-stage policies, and clinical semantics to constrain the symbolic reasoning space. We validate ArchMap on 1060 pre-/post-orthodontic cases, demonstrating robust performance in tooth counting, anatomical partitioning, dentition-stage classification, and the identification of clinical conditions such as crowding, missing teeth, prosthetics, and caries. Compared with supervised pipelines and prompted VLM baselines, ArchMap achieves higher accuracy, reduced semantic drift, and superior stability under sparse or artifact-prone conditions. As a fully training-free system, ArchMap demonstrates that combining geometric normalization with ontology-guided multimodal reasoning offers a practical and scalable solution for the structured analysis of 3D intraoral scans in modern digital orthodontics.

Method: Proposes DentalSCR framework with: 1) U-Midline Dental Axis (UMDA) for unified anatomical coordinate system, 2) radiograph-like projections using surface-based DRR with coronal-axis perspective and Gaussian splatting, 3) 2D similarity transform optimization with symmetric bidirectional Chamfer distance under hierarchical coarse-to-fine schedule.

Result: Evaluation on 34 clinical cases shows substantial reductions in landmark error (especially at posterior teeth), tighter dispersion on lower jaw, and low Chamfer and controlled Hausdorff distances at curve level, outperforming conventional baselines.

Conclusion: DentalSCR robustly handles real-world cephalograms and delivers high-fidelity, clinically inspectable 3D-2D alignment, providing a reliable solution for orthodontic diagnosis.

Abstract: Reliable 3D-2D alignment between intraoral scan (IOS) models and lateral cephalometric radiographs is critical for orthodontic diagnosis, yet conventional intensity-driven registration methods struggle under real clinical conditions, where cephalograms exhibit projective magnification, geometric distortion, low-contrast dental crowns, and acquisition-dependent variation. These factors hinder the stability of appearance-based similarity metrics and often lead to convergence failures or anatomically implausible alignments. To address these limitations, we propose DentalSCR, a pose-stable, contour-guided framework for accurate and interpretable silhouette-to-contour registration. Our method first constructs a U-Midline Dental Axis (UMDA) to establish a unified cross-arch anatomical coordinate system, thereby stabilizing initialization and standardizing projection geometry across cases. Using this reference frame, we generate radiograph-like projections via a surface-based DRR formulation with coronal-axis perspective and Gaussian splatting, which preserves clinical source-object-detector magnification and emphasizes external silhouettes. Registration is then formulated as a 2D similarity transform optimized with a symmetric bidirectional Chamfer distance under a hierarchical coarse-to-fine schedule, enabling both large capture range and subpixel-level contour agreement. We evaluate DentalSCR on 34 expert-annotated clinical cases. Experimental results demonstrate substantial reductions in landmark error-particularly at posterior teeth-tighter dispersion on the lower jaw, and low Chamfer and controlled Hausdorff distances at the curve level. These findings indicate that DentalSCR robustly handles real-world cephalograms and delivers high-fidelity, clinically inspectable 3D–2D alignment, outperforming conventional baselines.

Method: Curated a bilingual English-Chinese chapter dataset using a structured pipeline that unifies ASR transcripts, scene texts, and visual captions into multi-level annotations. Designed GRACE evaluation metric incorporating many-to-one segment overlaps and semantic similarity.

Result: Established new state-of-the-art by outperforming previous best by 14.0% in F1 score and 11.3% in SODA score. Showed clear performance improvements with data scaling in both volume and label intensity. Demonstrated excellent transferability, improving state-of-the-art on downstream tasks like dense video captioning.

Conclusion: ARC-Chapter represents a significant advancement in video chaptering through large-scale training and comprehensive evaluation metrics, enabling better content structuring for long videos with strong generalization capabilities.

Abstract: The proliferation of hour-long videos (e.g., lectures, podcasts, documentaries) has intensified demand for efficient content structuring. However, existing approaches are constrained by small-scale training with annotations that are typical short and coarse, restricting generalization to nuanced transitions in long videos. We introduce ARC-Chapter, the first large-scale video chaptering model trained on over million-level long video chapters, featuring bilingual, temporally grounded, and hierarchical chapter annotations. To achieve this goal, we curated a bilingual English-Chinese chapter dataset via a structured pipeline that unifies ASR transcripts, scene texts, visual captions into multi-level annotations, from short title to long summaries. We demonstrate clear performance improvements with data scaling, both in data volume and label intensity. Moreover, we design a new evaluation metric termed GRACE, which incorporates many-to-one segment overlaps and semantic similarity, better reflecting real-world chaptering flexibility. Extensive experiments demonstrate that ARC-Chapter establishes a new state-of-the-art by a significant margin, outperforming the previous best by 14.0% in F1 score and 11.3% in SODA score. Moreover, ARC-Chapter shows excellent transferability, improving the state-of-the-art on downstream tasks like dense video captioning on YouCook2.

Method: Model pixel color as combination of base color from standard 3DGS rendering and learned residual from neighboring training images, leveraging high-resolution source images.

Result: Significantly outperforms prior Gaussian Splatting approaches in rendering quality on standard novel view synthesis benchmarks.

Conclusion: The proposed method enables rendering of high-frequency details and accurate view-dependent effects while maintaining the same storage footprint as standard 3DGS.

Abstract: 3D Gaussian Splatting (3DGS) has recently emerged as a fast, high-quality method for novel view synthesis (NVS). However, its use of low-degree spherical harmonics limits its ability to capture spatially varying color and view-dependent effects such as specular highlights. Existing works augment Gaussians with either a global texture map, which struggles with complex scenes, or per-Gaussian texture maps, which introduces high storage overhead. We propose Image-Based Gaussian Splatting, an efficient alternative that leverages high-resolution source images for fine details and view-specific color modeling. Specifically, we model each pixel color as a combination of a base color from standard 3DGS rendering and a learned residual inferred from neighboring training images. This promotes accurate surface alignment and enables rendering images of high-frequency details and accurate view-dependent effects. Experiments on standard NVS benchmarks show that our method significantly outperforms prior Gaussian Splatting approaches in rendering quality, without increasing the storage footprint.

Method: Deep learning framework for WMH segmentation and localization, evaluated across public datasets and ADNI cohort, with regional WMH quantification and integration with brain structure volumes.

Result: Regional WMH volumes consistently outperform global lesion burden for disease classification; integration with atrophy metrics improves performance to AUC 0.97; anterior white matter tracts show reproducible association with AD diagnosis.

Conclusion: Regional WMH quantification provides added diagnostic value over global measures; combining localized lesion metrics with atrophy markers may enhance early diagnosis and stratification in neurodegenerative disorders.

Abstract: White matter hyperintensities (WMH) are key imaging markers in cognitive aging, Alzheimer’s disease (AD), and related dementias. Although automated methods for WMH segmentation have advanced, most provide only global lesion load and overlook their spatial distribution across distinct white matter regions. We propose a deep learning framework for robust WMH segmentation and localization, evaluated across public datasets and an independent Alzheimer’s Disease Neuroimaging Initiative (ADNI) cohort. Our results show that the predicted lesion loads are in line with the reference WMH estimates, confirming the robustness to variations in lesion load, acquisition, and demographics. Beyond accurate segmentation, we quantify WMH load within anatomically defined regions and combine these measures with brain structure volumes to assess diagnostic value. Regional WMH volumes consistently outperform global lesion burden for disease classification, and integration with brain atrophy metrics further improves performance, reaching area under the curve (AUC) values up to 0.97. Several spatially distinct regions, particularly within anterior white matter tracts, are reproducibly associated with diagnostic status, indicating localized vulnerability in AD. These results highlight the added value of regional WMH quantification. Incorporating localized lesion metrics alongside atrophy markers may enhance early diagnosis and stratification in neurodegenerative disorders.

Method: Dual-branch architecture with shared weights, lightweight feature restoration network using clear branch as supervision, frequency structure guidance module, and feature consistency self-supervised loss between branches.

Result: Extensive experiments on IRBlurUAV benchmark (30k simulated + 4,118 real images) show superior detection performance while maintaining real-time efficiency.

Conclusion: JFD3 effectively enhances discriminative feature representation for detection under blur conditions through joint feature-domain deblurring and detection approach.

Abstract: Infrared unmanned aerial vehicle (UAV) target images often suffer from motion blur degradation caused by rapid sensor movement, significantly reducing contrast between target and background. Generally, detection performance heavily depends on the discriminative feature representation between target and background. Existing methods typically treat deblurring as a preprocessing step focused on visual quality, while neglecting the enhancement of task-relevant features crucial for detection. Improving feature representation for detection under blur conditions remains challenging. In this paper, we propose a novel Joint Feature-Domain Deblurring and Detection end-to-end framework, dubbed JFD3. We design a dual-branch architecture with shared weights, where the clear branch guides the blurred branch to enhance discriminative feature representation. Specifically, we first introduce a lightweight feature restoration network, where features from the clear branch serve as feature-level supervision to guide the blurred branch, thereby enhancing its distinctive capability for detection. We then propose a frequency structure guidance module that refines the structure prior from the restoration network and integrates it into shallow detection layers to enrich target structural information. Finally, a feature consistency self-supervised loss is imposed between the dual-branch detection backbones, driving the blurred branch to approximate the feature representations of the clear one. Wealso construct a benchmark, named IRBlurUAV, containing 30,000 simulated and 4,118 real infrared UAV target images with diverse motion blur. Extensive experiments on IRBlurUAV demonstrate that JFD3 achieves superior detection performance while maintaining real-time efficiency.

Method: Quantifies shoulder yaw and roll using 3D coordinates of two shoulder landmarks from pose estimation frameworks, evaluated on 121 portrait images.

Result: SPE scores showed strong Pearson correlation (r ≈ 0.80) with human labels and effectively identified non-compliant samples using Error-versus-Discard analysis.

Conclusion: The SPE algorithm is a viable lightweight tool for automated compliance checking in enrollment systems for biometric documents.

Abstract: International standards for biometric identity documents mandate strict compliance with pose requirements, including the square presentation of a subject’s shoulders. However, the literature on automated quality assessment offers few quantitative methods for evaluating this specific attribute. This paper proposes a Shoulder Presentation Evaluation (SPE) algorithm to address this gap. The method quantifies shoulder yaw and roll using only the 3D coordinates of two shoulder landmarks provided by common pose estimation frameworks. The algorithm was evaluated on a dataset of 121 portrait images. The resulting SPE scores demonstrated a strong Pearson correlation (r approx. 0.80) with human-assigned labels. An analysis of the metric’s filtering performance, using an adapted Error-versus-Discard methodology, confirmed its utility in identifying non-compliant samples. The proposed algorithm is a viable lightweight tool for automated compliance checking in enrolment systems.

Method: TLS-Assist converts traffic light and sign detections into structured natural language messages that are injected into the LLM input, enforcing explicit attention to safety-critical cues. It’s plug-and-play, model-agnostic, and supports both single-view and multi-view camera setups.

Result: Evaluation on LangAuto benchmark in CARLA showed relative driving performance improvements of up to 14% over LMDrive and 7% over BEVDriver, while consistently reducing traffic light and sign infractions.

Conclusion: TLS-Assist effectively addresses the safety limitations of LLM-based autonomous driving agents by providing explicit traffic rule enforcement through structured detection messaging, significantly improving driving performance and reducing infractions.

Abstract: Large Language Models (LLMs) are increasingly used for decision-making and planning in autonomous driving, showing promising reasoning capabilities and potential to generalize across diverse traffic situations. However, current LLM-based driving agents lack explicit mechanisms to enforce traffic rules and often struggle to reliably detect small, safety-critical objects such as traffic lights and signs. To address this limitation, we introduce TLS-Assist, a modular redundancy layer that augments LLM-based autonomous driving agents with explicit traffic light and sign recognition. TLS-Assist converts detections into structured natural language messages that are injected into the LLM input, enforcing explicit attention to safety-critical cues. The framework is plug-and-play, model-agnostic, and supports both single-view and multi-view camera setups. We evaluate TLS-Assist in a closed-loop setup on the LangAuto benchmark in CARLA. The results demonstrate relative driving performance improvements of up to 14% over LMDrive and 7% over BEVDriver, while consistently reducing traffic light and sign infractions. We publicly release the code and models on https://github.com/iis-esslingen/TLS-Assist.

Method: Uses shared-specific encoder-decoder architecture with hierarchical modality self-distillation and progressive modality combination distillation to simulate gradual modality dropout during training.

Result: Achieves state-of-the-art performance on public brain tumor segmentation benchmarks across various missing-modality scenarios with strong generalization and stability.

Conclusion: CCSD effectively addresses the challenge of missing modalities in clinical MRI data through innovative self-distillation strategies.

Abstract: The accurate segmentation of brain tumors from multi-modal MRI is critical for clinical diagnosis and treatment planning. While integrating complementary information from various MRI sequences is a common practice, the frequent absence of one or more modalities in real-world clinical settings poses a significant challenge, severely compromising the performance and generalizability of deep learning-based segmentation models. To address this challenge, we propose a novel Cross-Modal Compositional Self-Distillation (CCSD) framework that can flexibly handle arbitrary combinations of input modalities. CCSD adopts a shared-specific encoder-decoder architecture and incorporates two self-distillation strategies: (i) a hierarchical modality self-distillation mechanism that transfers knowledge across modality hierarchies to reduce semantic discrepancies, and (ii) a progressive modality combination distillation approach that enhances robustness to missing modalities by simulating gradual modality dropout during training. Extensive experiments on public brain tumor segmentation benchmarks demonstrate that CCSD achieves state-of-the-art performance across various missing-modality scenarios, with strong generalization and stability.

Method: The authors created BEDLAM2.0, an improved dataset that expands on BEDLAM by adding more diverse and realistic cameras and camera motions, increasing diversity in body shape, motions, clothing, hair, and 3D environments, and including shoes which were missing in the original dataset.

Result: BEDLAM2.0 significantly improves accuracy over BEDLAM when training state-of-the-art methods for estimating humans in world coordinates. The dataset provides rendered videos, ground truth body parameters, camera motions, and 3D assets.

Conclusion: BEDLAM2.0 serves as a superior training resource compared to BEDLAM for 3D human pose and motion regressors, particularly for world coordinate estimation tasks, and is made available for research purposes.

Abstract: Inferring 3D human motion from video remains a challenging problem with many applications. While traditional methods estimate the human in image coordinates, many applications require human motion to be estimated in world coordinates. This is particularly challenging when there is both human and camera motion. Progress on this topic has been limited by the lack of rich video data with ground truth human and camera movement. We address this with BEDLAM2.0, a new dataset that goes beyond the popular BEDLAM dataset in important ways. In addition to introducing more diverse and realistic cameras and camera motions, BEDLAM2.0 increases diversity and realism of body shape, motions, clothing, hair, and 3D environments. Additionally, it adds shoes, which were missing in BEDLAM. BEDLAM has become a key resource for training 3D human pose and motion regressors today and we show that BEDLAM2.0 is significantly better, particularly for training methods that estimate humans in world coordinates. We compare state-of-the art methods trained on BEDLAM and BEDLAM2.0, and find that BEDLAM2.0 significantly improves accuracy over BEDLAM. For research purposes, we provide the rendered videos, ground truth body parameters, and camera motions. We also provide the 3D assets to which we have rights and links to those from third parties.

Method: Used Dynamic Time Warping clustering for trajectory-aware labeling, trained 3D Vision Transformer via unsupervised reconstruction on harmonized MRI data to obtain anatomy-preserving embeddings, and compared against tabular representations and convolutional baselines using traditional ML and deep learning classifiers.

Result: Clinical/volumetric features achieved AUC ~0.70 for predicting mild/severe progression, while ViT MRI embeddings achieved AUC 0.71 for distinguishing stable individuals. All methods struggled with moderate progression group.

Conclusion: Clinical features are best for identifying high-risk extremes, transformer-based MRI embeddings are more sensitive to subtle stability markers, motivating multimodal fusion strategies for comprehensive AD progression modeling.

Abstract: Accurate modeling of cognitive decline in Alzheimer’s disease is essential for early stratification and personalized management. While tabular predictors provide robust markers of global risk, their ability to capture subtle brain changes remains limited. In this study, we evaluate the predictive contributions of tabular and imaging-based representations, with a focus on transformer-derived Magnetic Resonance Imaging (MRI) embeddings. We introduce a trajectory-aware labeling strategy based on Dynamic Time Warping clustering to capture heterogeneous patterns of cognitive change, and train a 3D Vision Transformer (ViT) via unsupervised reconstruction on harmonized and augmented MRI data to obtain anatomy-preserving embeddings without progression labels. The pretrained encoder embeddings are subsequently assessed using both traditional machine learning classifiers and deep learning heads, and compared against tabular representations and convolutional network baselines. Results highlight complementary strengths across modalities. Clinical and volumetric features achieved the highest AUCs of around 0.70 for predicting mild and severe progression, underscoring their utility in capturing global decline trajectories. In contrast, MRI embeddings from the ViT model were most effective in distinguishing cognitively stable individuals with an AUC of 0.71. However, all approaches struggled in the heterogeneous moderate group. These findings indicate that clinical features excel in identifying high-risk extremes, whereas transformer-based MRI embeddings are more sensitive to subtle markers of stability, motivating multimodal fusion strategies for AD progression modeling.

Method: Ordinal Regression-based DR detection framework using APTOS-2019 fundus images with preprocessing (Green Channel Extraction, Noise Masking, CLAHE) to isolate relevant features.

Result: Achieved Quadratic Weighted Kappa score of 0.8992, setting a new benchmark on the APTOS dataset.

Conclusion: The proposed Ordinal Regression approach effectively detects Diabetic Retinopathy with high agreement to clinical grading, enabling timely intervention.

Abstract: Diabetic Retinopathy (DR) has emerged as a major cause of preventable blindness in recent times. With timely screening and intervention, the condition can be prevented from causing irreversible damage. The work introduces a state-of-the-art Ordinal Regression-based DR Detection framework that uses the APTOS-2019 fundus image dataset. A widely accepted combination of preprocessing methods: Green Channel (GC) Extraction, Noise Masking, and CLAHE, was used to isolate the most relevant features for DR classification. Model performance was evaluated using the Quadratic Weighted Kappa, with a focus on agreement between results and clinical grading. Our Ordinal Regression approach attained a QWK score of 0.8992, setting a new benchmark on the APTOS dataset.

Method: LAVA replaces direct feature alignment with relative alignment driven by text-based reference anchors. It guides visual representations to preserve consistent relative geometry defined by pairwise semantic similarities between class names.

Result: Extensive experiments on standard DIL benchmarks show LAVA achieves significant performance improvements over state-of-the-art methods.

Conclusion: LAVA successfully addresses the DIL dilemma by using language-anchored visual alignment to create a bridge across domains, enabling knowledge retrieval and robust feature aggregation without interference or fragmentation.

Abstract: A key challenge in Domain Incremental Learning (DIL) is to continually learn under shifting distributions while preserving knowledge from previous domains. Existing methods face a fundamental dilemma. On one hand, projecting all domains into a single unified visual space leads to inter-domain interference and semantic distortion, as large shifts may vary with not only visual appearance but also underlying semantics. On the other hand, isolating domain-specific parameters causes knowledge fragmentation, creating “knowledge islands” that hamper knowledge reuse and exacerbate forgetting. To address this issue, we propose LAVA (Language-Anchored Visual Alignment), a novel DIL framework that replaces direct feature alignment with relative alignment driven by a text-based reference anchor. LAVA guides the visual representations of each incoming domain to preserve a consistent relative geometry, which is defined by mirroring the pairwise semantic similarities between the class names. This anchored geometric structure acts as a bridge across domains, enabling the retrieval of class-aware prior knowledge and facilitating robust feature aggregation. Extensive experiments on standard DIL benchmarks demonstrate that LAVA achieves significant performance improvements over state-of-the-arts. Code is available at https://github.com/ShuyiGeng/LAVA.

Method: Proposed approach incorporates features from a pulse analysis pipeline into standard U-Net segmentation architectures to exploit temporal dynamics in Doppler holograms.

Result: The method achieves artery-vein segmentation performance comparable to more complex attention- or iteration-based models, demonstrating the effectiveness of temporal preprocessing.

Conclusion: Time-resolved preprocessing can unlock deep learning’s full potential for Doppler holography, enabling better quantitative analysis of retinal hemodynamics.

Abstract: Doppler holography is an emerging retinal imaging technique that captures the dynamic behavior of blood flow with high temporal resolution, enabling quantitative assessment of retinal hemodynamics. This requires accurate segmentation of retinal arteries and veins, but traditional segmentation methods focus solely on spatial information and overlook the temporal richness of holographic data. In this work, we propose a simple yet effective approach for artery-vein segmentation in temporal Doppler holograms using standard segmentation architectures. By incorporating features derived from a dedicated pulse analysis pipeline, our method allows conventional U-Nets to exploit temporal dynamics and achieve performance comparable to more complex attention- or iteration-based models. These findings demonstrate that time-resolved preprocessing can unlock the full potential of deep learning for Doppler holography, opening new perspectives for quantitative exploration of retinal hemodynamics. The dataset is publicly available at https://huggingface.co/datasets/DigitalHolography/

Method: 1) Automatic landmark annotation on 2D skull X-rays using trained YOLO-pose models; 2) Cross-modal matching by converting landmarks into graph representations and finding semantic correspondence using cross-attention and optimal transport framework.

Result: Extensive experiments on S2F and CUHK datasets demonstrate significant improvements in reliability and accuracy, showing effectiveness in cross-domain skull-to-face and sketch-to-face matching for forensic science.

Conclusion: The proposed Cranio-ID framework provides a reliable and accurate automated solution for craniometric landmark analysis and cross-modal matching in forensic identification applications.

Abstract: In forensic craniofacial identification and in many biomedical applications, craniometric landmarks are important. Traditional methods for locating landmarks are time-consuming and require specialized knowledge and expertise. Current methods utilize superimposition and deep learning-based methods that employ automatic annotation of landmarks. However, these methods are not reliable due to insufficient large-scale validation studies. In this paper, we proposed a novel framework Cranio-ID: First, an automatic annotation of landmarks on 2D skulls (which are X-ray scans of faces) with their respective optical images using our trained YOLO-pose models. Second, cross-modal matching by formulating these landmarks into graph representations and then finding semantic correspondence between graphs of these two modalities using cross-attention and optimal transport framework. Our proposed framework is validated on the S2F and CUHK datasets (CUHK dataset resembles with S2F dataset). Extensive experiments have been conducted to evaluate the performance of our proposed framework, which demonstrates significant improvements in both reliability and accuracy, as well as its effectiveness in cross-domain skull-to-face and sketch-to-face matching in forensic science.

Method: Train self-supervised learning models on object-centric videos under CATDiet constraints (Color, Acuity, Temporal continuity), then develop CombDiet which initializes SSL with CATDiet before standard training while preserving temporal continuity.

Result: CATDiet variants demonstrate enhanced robustness in object recognition across 10 datasets, show biologically aligned developmental patterns mirroring neural plasticity in macaque V1 and infant visual cliff responses. CombDiet outperforms standard SSL on both in-domain and out-of-domain object recognition and depth perception.

Conclusion: The developmental progression of early infant visual experience offers a powerful reverse-engineering framework for understanding the emergence of robust visual intelligence in machines.

Abstract: Newborns perceive the world with low-acuity, color-degraded, and temporally continuous vision, which gradually sharpens as infants develop. To explore the ecological advantages of such staged “visual diets”, we train self-supervised learning (SSL) models on object-centric videos under constraints that simulate infant vision: grayscale-to-color (C), blur-to-sharp (A), and preserved temporal continuity (T)-collectively termed CATDiet. For evaluation, we establish a comprehensive benchmark across ten datasets, covering clean and corrupted image recognition, texture-shape cue conflict tests, silhouette recognition, depth-order classification, and the visual cliff paradigm. All CATDiet variants demonstrate enhanced robustness in object recognition, despite being trained solely on object-centric videos. Remarkably, models also exhibit biologically aligned developmental patterns, including neural plasticity changes mirroring synaptic density in macaque V1 and behaviors resembling infants’ visual cliff responses. Building on these insights, CombDiet initializes SSL with CATDiet before standard training while preserving temporal continuity. Trained on object-centric or head-mounted infant videos, CombDiet outperforms standard SSL on both in-domain and out-of-domain object recognition and depth perception. Together, these results suggest that the developmental progression of early infant visual experience offers a powerful reverse-engineering framework for understanding the emergence of robust visual intelligence in machines. All code, data, and models will be publicly released.

Method: Used 1000 images from IMDB-Clean dataset processed in seven resolutions (7000 total samples), evaluated using MAE, SD, and MedAE metrics on DeepFace and InsightFace frameworks.

Result: Both frameworks achieve best performance at 224x224 pixels (MAE: 10.83 years for DeepFace, 7.46 years for InsightFace). Low and very high resolutions degrade accuracy. InsightFace is consistently faster.

Conclusion: Image resolution has clear and consistent impact on age estimation accuracy, with 224x224 being optimal for both frameworks.

Abstract: Automatic age estimation is widely used for age verification, where input images often vary considerably in resolution. This study evaluates the effect of image resolution on age estimation accuracy using DeepFace and InsightFace. A total of 1000 images from the IMDB-Clean dataset were processed in seven resolutions, resulting in 7000 test samples. Performance was evaluated using Mean Absolute Error (MAE), Standard Deviation (SD), and Median Absolute Error (MedAE). Based on this study, we conclude that input image resolution has a clear and consistent impact on the accuracy of age estimation in both DeepFace and InsightFace. Both frameworks achieve optimal performance at 224x224 pixels, with an MAE of 10.83 years (DeepFace) and 7.46 years (InsightFace). At low resolutions, MAE increases substantially, while very high resolutions also degrade accuracy. InsightFace is consistently faster than DeepFace across all resolutions.

Method: Uses two specialized modules: Dialog Refiner Module that queries users to extract essential information and generate precise descriptions, and Image Refiner Module that identifies perceptual gaps between generated images and user intentions to reduce visual-semantic discrepancy.

Result: Significantly improves target image hit accuracy compared to conventional single-query methods, with comprehensive experiments showing substantial performance gains over initial-description-only baselines across diverse image datasets.

Conclusion: The dialogue-based approach provides superior controllability and fault tolerance, with synergistic module integration achieving both higher retrieval precision and enhanced interactive experience.

Abstract: This paper addresses the task of interactive, conversational text-to-image retrieval. Our DIR-TIR framework progressively refines the target image search through two specialized modules: the Dialog Refiner Module and the Image Refiner Module. The Dialog Refiner actively queries users to extract essential information and generate increasingly precise descriptions of the target image. Complementarily, the Image Refiner identifies perceptual gaps between generated images and user intentions, strategically reducing the visual-semantic discrepancy. By leveraging multi-turn dialogues, DIR-TIR provides superior controllability and fault tolerance compared to conventional single-query methods, significantly improving target image hit accuracy. Comprehensive experiments across diverse image datasets demonstrate our dialogue-based approach substantially outperforms initial-description-only baselines, while the synergistic module integration achieves both higher retrieval precision and enhanced interactive experience.

Method: CompEvent uses two core components: Complex Temporal Alignment GRU for temporal alignment and continuous fusion of video and event streams, and Complex Space-Frequency Learning module for unified complex-valued signal processing in spatial and frequency domains.

Result: Extensive experiments demonstrate that CompEvent outperforms state-of-the-art methods in low-light video deblurring, achieving superior performance through full-process spatiotemporal fusion.

Conclusion: By leveraging complex-valued neural networks’ holistic representation capability, CompEvent maximizes complementary learning between modalities and significantly strengthens low-light video deblurring capability.

Abstract: Low-light video deblurring poses significant challenges in applications like nighttime surveillance and autonomous driving due to dim lighting and long exposures. While event cameras offer potential solutions with superior low-light sensitivity and high temporal resolution, existing fusion methods typically employ staged strategies, limiting their effectiveness against combined low-light and motion blur degradations. To overcome this, we propose CompEvent, a complex neural network framework enabling holistic full-process fusion of event data and RGB frames for enhanced joint restoration. CompEvent features two core components: 1) Complex Temporal Alignment GRU, which utilizes complex-valued convolutions and processes video and event streams iteratively via GRU to achieve temporal alignment and continuous fusion; and 2) Complex Space-Frequency Learning module, which performs unified complex-valued signal processing in both spatial and frequency domains, facilitating deep fusion through spatial structures and system-level characteristics. By leveraging the holistic representation capability of complex-valued neural networks, CompEvent achieves full-process spatiotemporal fusion, maximizes complementary learning between modalities, and significantly strengthens low-light video deblurring capability. Extensive experiments demonstrate that CompEvent outperforms SOTA methods in addressing this challenging task. The code is available at https://github.com/YuXie1/CompEvent.

Method: DeepLabV3+ decoder with ResNet-50 encoder trained with physics constraints including multi-scale mass conservation, flow-aligned total variation, Laplacian damping, non-negativity of thickness, prior-consistency term, and masked Huber fit to radar picks with confidence modulation.

Result: Achieves strong test-core accuracy and high structural fidelity across two Greenland sub-regions, outperforming U-Net, Attention U-Net, FPN, and plain CNN architectures.

Conclusion: The residual-over-prior design combined with physics constraints produces spatially coherent and physically plausible bed topography suitable for operational mapping under domain shift conditions.

Abstract: Accurate subglacial bed topography is essential for ice sheet modeling, yet radar observations are sparse and uneven. We propose a physics-guided residual learning framework that predicts bed thickness residuals over a BedMachine prior and reconstructs bed from the observed surface. A DeepLabV3+ decoder over a standard encoder (e.g.,ResNet-50) is trained with lightweight physics and data terms: multi-scale mass conservation, flow-aligned total variation, Laplacian damping, non-negativity of thickness, a ramped prior-consistency term, and a masked Huber fit to radar picks modulated by a confidence map. To measure real-world generalization, we adopt leakage-safe blockwise hold-outs (vertical/horizontal) with safety buffers and report metrics only on held-out cores. Across two Greenland sub-regions, our approach achieves strong test-core accuracy and high structural fidelity, outperforming U-Net, Attention U-Net, FPN, and a plain CNN. The residual-over-prior design, combined with physics, yields spatially coherent, physically plausible beds suitable for operational mapping under domain shift.

Method: Proposed multimodal framework with Temporal Aware Feature Fusion (TAFF) mechanism that dynamically weights and fuses ECG-derived electrophysiological features with anatomical priors from AHA-17 atlas, accounting for acquisition time differences between ECGs and LGE-MRIs.

Result: Achieved substantial improvement over state-of-the-art image-only baseline (nnU-Net), increasing average Dice score for scars from 0.6149 to 0.8463, with high precision (0.9115) and sensitivity (0.9043).

Conclusion: Integrating physiological and anatomical knowledge enables the model to ‘see beyond the image’, setting a new direction for robust and physiologically grounded cardiac scar segmentation.

Abstract: Accurate segmentation of myocardial scar from late gadolinium enhanced (LGE) cardiac MRI is essential for evaluating tissue viability, yet remains challenging due to variable contrast and imaging artifacts. Electrocardiogram (ECG) signals provide complementary physiological information, as conduction abnormalities can help localize or suggest scarred myocardial regions. In this work, we propose a novel multimodal framework that integrates ECG-derived electrophysiological information with anatomical priors from the AHA-17 atlas for physiologically consistent LGE-based scar segmentation. As ECGs and LGE-MRIs are not acquired simultaneously, we introduce a Temporal Aware Feature Fusion (TAFF) mechanism that dynamically weights and fuses features based on their acquisition time difference. Our method was evaluated on a clinical dataset and achieved substantial gains over the state-of-the-art image-only baseline (nnU-Net), increasing the average Dice score for scars from 0.6149 to 0.8463 and achieving high performance in both precision (0.9115) and sensitivity (0.9043). These results show that integrating physiological and anatomical knowledge allows the model to “see beyond the image”, setting a new direction for robust and physiologically grounded cardiac scar segmentation.

Method: Proposes Gaussian splatting-based method to estimate pixel-annotation correspondence, computes transport plan using Bayesian probability, and derives a loss function that measures discrepancy after transport for network optimization.

Result: Extensive experiments on crowd counting and landmark detection tasks validate the effectiveness of GST, showing it eliminates iterative transport plan computation while maintaining performance.

Conclusion: GST provides an efficient framework for spatial transport in computer vision tasks, significantly improving training efficiency compared to conventional optimal transport schemes while maintaining effectiveness.

Abstract: This paper introduces Gaussian Spatial Transport (GST), a novel framework that leverages Gaussian splatting to facilitate transport from the probability measure in the image coordinate space to the annotation map. We propose a Gaussian splatting-based method to estimate pixel-annotation correspondence, which is then used to compute a transport plan derived from Bayesian probability. To integrate the resulting transport plan into standard network optimization in typical computer vision tasks, we derive a loss function that measures discrepancy after transport. Extensive experiments on representative computer vision tasks, including crowd counting and landmark detection, validate the effectiveness of our approach. Compared to conventional optimal transport schemes, GST eliminates iterative transport plan computation during training, significantly improving efficiency. Code is available at https://github.com/infinite0522/GST.

Method: Uses a diffusion video foundational model conditioned on structure-aware information (depth maps, semantic maps, edge maps) extracted from synthetic videos via an auxiliary model, preserving spatial and temporal structures.

Result: Outperforms existing baselines in structural consistency while maintaining state-of-the-art photorealism quality.

Conclusion: Proposed approach is a simple yet general and powerful method for synthetic video realism enhancement that effectively preserves multi-level structures.

Abstract: We propose an approach to enhancing synthetic video realism, which can re-render synthetic videos from a simulator in photorealistic fashion. Our realism enhancement approach is a zero-shot framework that focuses on preserving the multi-level structures from synthetic videos into the enhanced one in both spatial and temporal domains, built upon a diffusion video foundational model without further fine-tuning. Specifically, we incorporate an effective modification to have the generation/denoising process conditioned on estimated structure-aware information from the synthetic video, such as depth maps, semantic maps, and edge maps, by an auxiliary model, rather than extracting the information from a simulator. This guidance ensures that the enhanced videos are consistent with the original synthetic video at both the structural and semantic levels. Our approach is a simple yet general and powerful approach to enhancing synthetic video realism: we show that our approach outperforms existing baselines in structural consistency with the original video while maintaining state-of-the-art photorealism quality in our experiments.

Method: Uses conditional latent diffusion models and large-scale multi-spectral, multi-source geo-spatial foundation models to bypass explicit super-resolution in pixel space, instead performing super-resolution directly in a segmentation-aware latent space.

Result: Achieves up to 25.5% relative improvement in instance segmentation and 12.9% relative improvement in semantic segmentation metrics over state-of-the-art Ref-SR methods on two large real datasets, with unprecedented 20x scale factor capability.

Conclusion: The segmentation-aware latent space approach enables effective combination of high-resolution reference imagery with low-resolution frequent imagery for agricultural monitoring, significantly outperforming existing methods for farm boundary delineation tasks.

Abstract: Delineating farm boundaries through segmentation of satellite images is a fundamental step in many agricultural applications. The task is particularly challenging for smallholder farms, where accurate delineation requires the use of high resolution (HR) imagery which are available only at low revisit frequencies (e.g., annually). To support more frequent (sub-) seasonal monitoring, HR images could be combined as references (ref) with low resolution (LR) images – having higher revisit frequency (e.g., weekly) – using reference-based super-resolution (Ref-SR) methods. However, current Ref-SR methods optimize perceptual quality and smooth over crucial features needed for downstream tasks, and are unable to meet the large scale-factor requirements for this task. Further, previous two-step approaches of SR followed by segmentation do not effectively utilize diverse satellite sources as inputs. We address these problems through a new approach, $\textbf{SEED-SR}$, which uses a combination of conditional latent diffusion models and large-scale multi-spectral, multi-source geo-spatial foundation models. Our key innovation is to bypass the explicit SR task in the pixel space and instead perform SR in a segmentation-aware latent space. This unique approach enables us to generate segmentation maps at an unprecedented 20$\times$ scale factor, and rigorous experiments on two large, real datasets demonstrate up to $\textbf{25.5}$ and $\textbf{12.9}$ relative improvement in instance and semantic segmentation metrics respectively over approaches based on state-of-the-art Ref-SR methods.

Method: Introduce Risk Semantic Distillation (RSD) with RiskHead module that distills causal risk estimates from Vision-Language Models into Bird’s-Eye-View features.

Result: Significant improvement in perception and planning capabilities on Bench2Drive benchmark, with enhanced ability to handle spatial boundaries and risky objects.

Conclusion: RSD effectively enhances autonomous driving generalization by learning richer risk attention representations that align with human-like driving behavior.

Abstract: The autonomous driving (AD) system has exhibited remarkable performance in complex driving scenarios. However, generalization is still a key limitation for the current system, which refers to the ability to handle unseen scenarios or unfamiliar sensor configurations.Related works have explored the use of Vision-Language Models (VLMs) to address few-shot or zero-shot tasks. While promising, these methods introduce a new challenge: the emergence of a hybrid AD system, where two distinct systems are used to plan a trajectory, leading to potential inconsistencies. Alternative research directions have explored Vision-Language-Action (VLA) frameworks that generate control actions from VLM directly. However, these end-to-end solutions demonstrate prohibitive computational demands. To overcome these challenges, we introduce Risk Semantic Distillation (RSD), a novel framework that leverages VLMs to enhance the training of End-to-End (E2E) AD backbones. By providing risk attention for key objects, RSD addresses the issue of generalization. Specifically, we introduce RiskHead, a plug-in module that distills causal risk estimates from Vision-Language Models into Bird’s-Eye-View (BEV) features, yielding interpretable risk-attention maps.This approach allows BEV features to learn richer and more nuanced risk attention representations, which directly enhance the model’s ability to handle spatial boundaries and risky objects.By focusing on risk attention, RSD aligns better with human-like driving behavior, which is essential to navigate in complex and dynamic environments. Our experiments on the Bench2Drive benchmark demonstrate the effectiveness of RSD in managing complex and unpredictable driving conditions. Due to the enhanced BEV representations enabled by RSD, we observed a significant improvement in both perception and planning capabilities.

Method: Represent ARC inputs on a canvas as natural images, then apply Vision Transformers for image-to-image mapping. The model is trained from scratch on ARC data and uses test-time training for generalization to unseen tasks.

Result: Achieves 60.4% accuracy on ARC-1 benchmark, substantially outperforming existing from-scratch methods and being competitive with leading LLMs while closing the gap to average human performance.

Conclusion: Vision-centric approaches to ARC are highly effective, demonstrating that treating abstract reasoning as an image-to-image translation problem can achieve state-of-the-art results comparable to human performance.

Abstract: The Abstraction and Reasoning Corpus (ARC) is designed to promote research on abstract reasoning, a fundamental aspect of human intelligence. Common approaches to ARC treat it as a language-oriented problem, addressed by large language models (LLMs) or recurrent reasoning models. However, although the puzzle-like tasks in ARC are inherently visual, existing research has rarely approached the problem from a vision-centric perspective. In this work, we formulate ARC within a vision paradigm, framing it as an image-to-image translation problem. To incorporate visual priors, we represent the inputs on a “canvas” that can be processed like natural images. It is then natural for us to apply standard vision architectures, such as a vanilla Vision Transformer (ViT), to perform image-to-image mapping. Our model is trained from scratch solely on ARC data and generalizes to unseen tasks through test-time training. Our framework, termed Vision ARC (VARC), achieves 60.4% accuracy on the ARC-1 benchmark, substantially outperforming existing methods that are also trained from scratch. Our results are competitive with those of leading LLMs and close the gap to average human performance.

Method: Uses Parameter Aware Mamba Model with dual state space parameter experts to capture task-specific priors, employs S4 for global integration, and Multi-Directional Hilbert Scanning for 2D feature sequences.

Result: Extensive experiments on NYUD-v2 and PASCAL-Context benchmarks demonstrate the method’s effectiveness.

Conclusion: PAMM provides an effective framework for multi-task dense prediction by leveraging state space models to enhance task interconnectivity and capture task-specific properties.

Abstract: Understanding the inter-relations and interactions between tasks is crucial for multi-task dense prediction. Existing methods predominantly utilize convolutional layers and attention mechanisms to explore task-level interactions. In this work, we introduce a novel decoder-based framework, Parameter Aware Mamba Model (PAMM), specifically designed for dense prediction in multi-task learning setting. Distinct from approaches that employ Transformers to model holistic task relationships, PAMM leverages the rich, scalable parameters of state space models to enhance task interconnectivity. It features dual state space parameter experts that integrate and set task-specific parameter priors, capturing the intrinsic properties of each task. This approach not only facilitates precise multi-task interactions but also allows for the global integration of task priors through the structured state space sequence model (S4). Furthermore, we employ the Multi-Directional Hilbert Scanning method to construct multi-angle feature sequences, thereby enhancing the sequence model’s perceptual capabilities for 2D data. Extensive experiments on the NYUD-v2 and PASCAL-Context benchmarks demonstrate the effectiveness of our proposed method. Our code is available at https://github.com/CQC-gogopro/PAMM.

Method: Two main components: 1) Visual Dual-path Extractor (ViDex) integrates DINOv2 semantic priors with local features for semantic-aware enhancement; 2) Global-Local State-Space block captures long-range and multi-scale features. Uses novel Deep Perceptual Relevancy Loss Function inspired by human contrast sensitivity.

Result: Extensive experiments on Mayo2016 dataset show D-PerceptCT outperforms state-of-the-art methods in preserving structural and textural information in LDCT images.

Conclusion: D-PerceptCT effectively enhances LDCT images by incorporating human visual system principles, providing radiologists with perceptually visible critical anatomical structures and pathological details.

Abstract: Low Dose Computed Tomography (LDCT) is widely used as an imaging solution to aid diagnosis and other clinical tasks. However, this comes at the price of a deterioration in image quality due to the low dose of radiation used to reduce the risk of secondary cancer development. While some efficient methods have been proposed to enhance LDCT quality, many overestimate noise and perform excessive smoothing, leading to a loss of critical details. In this paper, we introduce D-PerceptCT, a novel architecture inspired by key principles of the Human Visual System (HVS) to enhance LDCT images. The objective is to guide the model to enhance or preserve perceptually relevant features, thereby providing radiologists with CT images where critical anatomical structures and fine pathological details are perceptu- ally visible. D-PerceptCT consists of two main blocks: 1) a Visual Dual-path Extractor (ViDex), which integrates semantic priors from a pretrained DINOv2 model with local spatial features, allowing the network to incorporate semantic-awareness during enhancement; (2) a Global-Local State-Space block that captures long-range information and multiscale features to preserve the important structures and fine details for diagnosis. In addition, we propose a novel deep perceptual loss, designated as the Deep Perceptual Relevancy Loss Function (DPRLF), which is inspired by human contrast sensitivity, to further emphasize perceptually important features. Extensive experiments on the Mayo2016 dataset demonstrate the effectiveness of D-PerceptCT method for LDCT enhancement, showing better preservation of structural and textural information within LDCT images compared to SOTA methods.

Method: Developed VisAidMath benchmark for geometric problem-solving requiring visual aid construction, and introduced Three-Layered Funnel Evaluation Framework to assess accuracy, valid visual aid generation, and reasoning soundness.

Result: Experiments on state-of-the-art models show high surface-level accuracy conceals catastrophic failures in producing valid visual aids and reasoning from them, exposing a fundamental gap between visual perception and logical deduction.

Conclusion: Modern LMMs suffer from a ‘Reasoning Illusion’ where current evaluation metrics fail to capture their inability to strategically modify visual information for complex reasoning tasks.

Abstract: A hallmark of advanced artificial intelligence is the capacity to progress from passive visual perception to the strategic modification of visual information to facilitate complex reasoning. This advanced capability, however, remains critically underdeveloped in current Large Multi-modal Models (LMMs). The deficiency is often masked by evaluation metrics that prioritize final-answer accuracy, creating an illusion of competence where genuine reasoning is absent. Using the domain of geometric problem-solving as a precise instrument, we probe this issue through tasks that require constructing visual aids. To this end, we introduce \textbf{VisAidMath}, a challenging benchmark, and our novel Three-Layered Funnel Evaluation Framework. This framework moves beyond simple accuracy (ACCU) to scrutinize the generation of valid visual aids (PVA) and the soundness of subsequent reasoning steps (SPRS). Our extensive experiments on state-of-the-art models, including Doubao-Seed-1.6 and o4, reveal a profound ``Reasoning Illusion’’. We observe that high surface-level accuracy conceals a catastrophic failure in the models’ ability to produce valid visual aids or to reason from them. Our findings expose a fundamental schism between visual perception and logical deduction in modern LMMs. We host an evaluation platform at CodaBench for testing publicly. Homepage: https://nlp2ct.github.io/VisAidMathHomepage/ Evaluation: https://www.codabench.org/competitions/7634/

Method: Establishes a generative data framework based on unpaired image-to-image translation, producing a large-scale dataset covering 6 representative underwater degradation types, which constructs synthetic datasets with precise ground-truth labels.

Result: Models trained on the synthetic data achieve comparable or superior color restoration and generalization performance to those trained on existing benchmarks across 6 network architectures and 3 independent test sets.

Conclusion: Provides a reliable and scalable data-driven solution for underwater image restoration and enhancement, with the generated dataset publicly available.

Abstract: Underwater image restoration and enhancement are crucial for correcting color distortion and restoring image details, thereby establishing a fundamental basis for subsequent underwater visual tasks. However, current deep learning methodologies in this area are frequently constrained by the scarcity of high-quality paired datasets. Since it is difficult to obtain pristine reference labels in underwater scenes, existing benchmarks often rely on manually selected results from enhancement algorithms, providing debatable reference images that lack globally consistent color and authentic supervision. This limits the model’s capabilities in color restoration, image enhancement, and generalization. To overcome this limitation, we propose using in-air natural images as unambiguous reference targets and translating them into underwater-degraded versions, thereby constructing synthetic datasets that provide authentic supervision signals for model learning. Specifically, we establish a generative data framework based on unpaired image-to-image translation, producing a large-scale dataset that covers 6 representative underwater degradation types. The framework constructs synthetic datasets with precise ground-truth labels, which facilitate the learning of an accurate mapping from degraded underwater images to their pristine scene appearances. Extensive quantitative and qualitative experiments across 6 representative network architectures and 3 independent test sets show that models trained on our synthetic data achieve comparable or superior color restoration and generalization performance to those trained on existing benchmarks. This research provides a reliable and scalable data-driven solution for underwater image restoration and enhancement. The generated dataset is publicly available at: https://github.com/yftian2025/SynUIEDatasets.git.

Method: Represent multiple SDFs in common space using hybrid approach combining generalization-based and overfitting-based strategies. Introduces novel sampling strategy to improve training efficiency and eliminate artifacts from other SDFs influence.

Result: Numerical and visual evaluations on benchmarks show advantages over latest methods in representative ability and compactness. Achieves high-fidelity geometry detail preservation with compact latent representations.

Conclusion: The proposed method successfully overcomes limitations of neural SDFs for multiple shapes by combining learning strategies and novel sampling, enabling high-fidelity geometry detail recovery with compact latent codes.

Abstract: Neural signed distance functions (SDFs) have been a vital representation to represent 3D shapes or scenes with neural networks. An SDF is an implicit function that can query signed distances at specific coordinates for recovering a 3D surface. Although implicit functions work well on a single shape or scene, they pose obstacles when analyzing multiple SDFs with high-fidelity geometry details, due to the limited information encoded in the latent space for SDFs and the loss of geometry details. To overcome these obstacles, we introduce a method to represent multiple SDFs in a common space, aiming to recover more high-fidelity geometry details with more compact latent representations. Our key idea is to take full advantage of the benefits of generalization-based and overfitting-based learning strategies, which manage to preserve high-fidelity geometry details with compact latent codes. Based on this framework, we also introduce a novel sampling strategy to sample training queries. The sampling can improve the training efficiency and eliminate artifacts caused by the influence of other SDFs. We report numerical and visual evaluations on widely used benchmarks to validate our designs and show advantages over the latest methods in terms of the representative ability and compactness.

Method: Proposes interaction-aware hand-object Gaussians with optimizable parameters, incorporates hand information into object deformation fields, uses progressive optimization strategy, and applies explicit regularizations for stable representations.

Result: The approach surpasses existing dynamic 3D-GS-based methods and achieves state-of-the-art performance in reconstructing dynamic hand-object interaction scenes.

Conclusion: The proposed method effectively addresses the challenges of modeling hand-object interactions without object priors, achieving superior reconstruction quality through interaction-aware representations and progressive optimization.

Abstract: This paper focuses on a challenging setting of simultaneously modeling geometry and appearance of hand-object interaction scenes without any object priors. We follow the trend of dynamic 3D Gaussian Splatting based methods, and address several significant challenges. To model complex hand-object interaction with mutual occlusion and edge blur, we present interaction-aware hand-object Gaussians with newly introduced optimizable parameters aiming to adopt piecewise linear hypothesis for clearer structural representation. Moreover, considering the complementarity and tightness of hand shape and object shape during interaction dynamics, we incorporate hand information into object deformation field, constructing interaction-aware dynamic fields to model flexible motions. To further address difficulties in the optimization process, we propose a progressive strategy that handles dynamic regions and static background step by step. Correspondingly, explicit regularizations are designed to stabilize the hand-object representations for smooth motion transition, physical interaction reality, and coherent lighting. Experiments show that our approach surpasses existing dynamic 3D-GS-based methods and achieves state-of-the-art performance in reconstructing dynamic hand-object interaction.

Method: Uses three branches: RGB (ConvNeXt-tiny) for visual inconsistencies, texture (Swin Transformer-tiny) for blending artifacts, frequency (CNN + SE) for spectral noise. Includes attention-based temporal pooling and adaptive attention fusion.

Result: Achieves AUC 0.9752, F1-Score 0.9408, and accuracy 0.9208 on Celeb-DF (v2), outperforming single-stream baselines.

Conclusion: Comprehensive multi-modal feature fusion provides superior resilience against subtle forgeries, with ablation studies confirming branch synergy.

Abstract: Deepfakes generated by advanced GANs and autoencoders severely threaten information integrity and societal stability. Single-stream CNNs fail to capture multi-scale forgery artifacts across spatial, texture, and frequency domains, limiting robustness and generalization. We introduce the ForensicFlow, a tri-modal forensic framework that synergistically fuses RGB, texture, and frequency evidence for video Deepfake detection. The RGB branch (ConvNeXt-tiny) extracts global visual inconsistencies; the texture branch (Swin Transformer-tiny) detects fine-grained blending artifacts; the frequency branch (CNN + SE) identifies periodic spectral noise. Attention-based temporal pooling dynamically prioritizes high-evidence frames, while adaptive attention fusion balances branch contributions.Trained on Celeb-DF (v2) with Focal Loss, ForensicFlow achieves AUC 0.9752, F1-Score 0.9408, and accuracy 0.9208, outperforming single-stream baselines. Ablation validates branch synergy; Grad-CAM confirms forensic focus. This comprehensive feature fusion provides superior resilience against subtle forgeries.

Method: Clustering-based explainability technique that groups and visualizes model behaviors across multiple cases, providing both global and fine-grained insights.

Result: Successfully applied to a prostate cancer detection model, demonstrating the technique’s usefulness for model understanding and clinical confidence.

Conclusion: The clustering approach offers superior explainability compared to traditional saliency methods, enabling better model interpretation and faster clinical adoption of AI in pathology.

Abstract: We present a clustering-based explainability technique for digital pathology models based on convolutional neural networks. Unlike commonly used methods based on saliency maps, such as occlusion, GradCAM, or relevance propagation, which highlight regions that contribute the most to the prediction for a single slide, our method shows the global behaviour of the model under consideration, while also providing more fine-grained information. The result clusters can be visualised not only to understand the model, but also to increase confidence in its operation, leading to faster adoption in clinical practice. We also evaluate the performance of our technique on an existing model for detecting prostate cancer, demonstrating its usefulness.

Method: OmniZip identifies salient audio tokens, computes audio retention scores to capture information density, dynamically guides video token pruning using audio anchors enhanced by cross-modal similarity, and compresses video tokens using interleaved spatio-temporal scheme.

Result: Achieves 3.42X inference speedup and 1.4X memory reduction over other top-performing counterparts while maintaining performance without training.

Conclusion: OmniZip effectively bridges the gap in multimodal token compression for omnimodal LLMs, providing significant computational efficiency gains while preserving model performance.

Abstract: Omnimodal large language models (OmniLLMs) have attracted increasing research attention of late towards unified audio-video understanding, wherein processing audio-video token sequences creates a significant computational bottleneck, however. Existing token compression methods have yet to accommodate this emerging need of jointly compressing multimodal tokens. To bridge this gap, we present OmniZip, a training-free, audio-guided audio-visual token-compression framework that optimizes multimodal token representation and accelerates inference. Specifically, OmniZip first identifies salient audio tokens, then computes an audio retention score for each time group to capture information density, thereby dynamically guiding video token pruning and preserving cues from audio anchors enhanced by cross-modal similarity. For each time window, OmniZip compresses the video tokens using an interleaved spatio-temporal scheme. Extensive empirical results demonstrate the merits of OmniZip - it achieves 3.42X inference speedup and 1.4X memory reduction over other top-performing counterparts, while maintaining performance with no training.

Method: Uses bidirectional cross-attention mechanism to dynamically integrate hip X-ray images and clinical metadata features, with a Weighted Smooth L1 loss to address BMD imbalance and prioritize clinically significant cases.

Result: Outperforms baseline models, reducing MSE by 16.7%, MAE by 6.03%, and increasing R2 score by 16.4%. Cross-attention fusion significantly improves performance over naive feature concatenation.

Conclusion: The multimodal approach with cross-attention effectively estimates femoral neck BMD and shows potential for real-world clinical screening applications.

Abstract: Poor bone health is a significant public health concern, and low bone mineral density (BMD) leads to an increased fracture risk, a key feature of osteoporosis. We present XAttn-BMD (Cross-Attention BMD), a multimodal deep learning framework that predicts femoral neck BMD from hip X-ray images and structured clinical metadata. It utilizes a novel bidirectional cross-attention mechanism to dynamically integrate image and metadata features for cross-modal mutual reinforcement. A Weighted Smooth L1 loss is tailored to address BMD imbalance and prioritize clinically significant cases. Extensive experiments on the data from the Hertfordshire Cohort Study show that our model outperforms the baseline models in regression generalization and robustness. Ablation studies confirm the effectiveness of both cross-attention fusion and the customized loss function. Experimental results show that the integration of multimodal data via cross-attention outperforms naive feature concatenation without cross-attention, reducing MSE by 16.7%, MAE by 6.03%, and increasing the R2 score by 16.4%, highlighting the effectiveness of the approach for femoral neck BMD estimation. Furthermore, screening performance was evaluated using binary classification at clinically relevant femoral neck BMD thresholds, demonstrating the model’s potential in real-world scenarios.

Method: A 3D-aware flow-matching framework that retrieves morphologically corresponding spots on neighboring slides in shared feature space, fuses cross-section context into a lightweight ControlNet, uses a 3D-consistent prior combining learned ZINB prior with spatial-empirical prior, and employs global attention for linear scaling with spot count.

Result: HoloTea consistently improves 3D expression accuracy and generalization compared to 2D and 3D baselines across three spatial transcriptomics datasets spanning different tissue types and resolutions.

Conclusion: HoloTea advances the creation of accurate 3D virtual tissues, accelerating biomarker discovery and deepening understanding of disease through improved 3D-aware gene expression imputation from histology.

Abstract: A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease. Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well. We present Holographic Tissue Expression Inpainting and Analysis (HoloTea), a 3D-aware flow-matching framework that imputes spot-level gene expression from H&E while explicitly using information from adjacent sections. Our key idea is to retrieve morphologically corresponding spots on neighboring slides in a shared feature space and fuse this cross section context into a lightweight ControlNet, allowing conditioning to follow anatomical continuity. To better capture the count nature of the data, we introduce a 3D-consistent prior for flow matching that combines a learned zero-inflated negative binomial (ZINB) prior with a spatial-empirical prior constructed from neighboring sections. A global attention block introduces 3D H&E scaling linearly with the number of spots in the slide, enabling training and inference on large 3D ST datasets. Across three spatial transcriptomics datasets spanning different tissue types and resolutions, HoloTea consistently improves 3D expression accuracy and generalization compared to 2D and 3D baselines. We envision HoloTea advancing the creation of accurate 3D virtual tissues, ultimately accelerating biomarker discovery and deepening our understanding of disease.

Method: Proposed Biomechanical Spatio-Temporal Graph Convolutional Network (BioST-GCN) - a dual-stream model combining pose and biomechanical information with cross-attention fusion mechanism.

Result: Model outperformed vanilla ST-GCN by 5.32% and 2.91% F1-score on simulated datasets, achieving 89.0% F1-score with full supervision. However, zero-shot generalization to unseen subjects dropped to 35.9%, revealing significant simulation-reality gap.

Conclusion: There’s an urgent need to bridge the gap between simulated and real-world data through personalization strategies and privacy-preserving data pipelines to develop effective fall prediction systems for elderly populations.

Abstract: Falls are a leading cause of injury and loss of independence among older adults. Vision-based fall prediction systems offer a non-invasive solution to anticipate falls seconds before impact, but their development is hindered by the scarcity of available fall data. Contributing to these efforts, this study proposes the Biomechanical Spatio-Temporal Graph Convolutional Network (BioST-GCN), a dual-stream model that combines both pose and biomechanical information using a cross-attention fusion mechanism. Our model outperforms the vanilla ST-GCN baseline by 5.32% and 2.91% F1-score on the simulated MCF-UA stunt-actor and MUVIM datasets, respectively. The spatio-temporal attention mechanisms in the ST-GCN stream also provide interpretability by identifying critical joints and temporal phases. However, a critical simulation-reality gap persists. While our model achieves an 89.0% F1-score with full supervision on simulated data, zero-shot generalization to unseen subjects drops to 35.9%. This performance decline is likely due to biases in simulated data, such as `intent-to-fall’ cues. For older adults, particularly those with diabetes or frailty, this gap is exacerbated by their unique kinematic profiles. To address this, we propose personalization strategies and advocate for privacy-preserving data pipelines to enable real-world validation. Our findings underscore the urgent need to bridge the gap between simulated and real-world data to develop effective fall prediction systems for vulnerable elderly populations.

Method: Introduces Stereo Geometry-Texture Alignment to bridge rendering quality and geometry estimation, and presents Pseudo-Feature Enhanced Geometry Consistency that enforces multi-view geometric consistency using both training and unseen views.

Result: Achieves state-of-the-art performance on DTU, BlendedMVS, and Mip-NeRF360 datasets, demonstrating more accurate and detailed surface reconstruction while preserving high-quality novel view rendering.

Conclusion: The proposed method effectively addresses overfitting in sparse-view Gaussian Splatting by jointly enhancing surface reconstruction and view synthesis through geometry-texture alignment and multi-view consistency enforcement.

Abstract: Recent advances in optimizing Gaussian Splatting for scene geometry have enabled efficient reconstruction of detailed surfaces from images. However, when input views are sparse, such optimization is prone to overfitting, leading to suboptimal reconstruction quality. Existing approaches address this challenge by employing flattened Gaussian primitives to better fit surface geometry, combined with depth regularization to alleviate geometric ambiguities under limited viewpoints. Nevertheless, the increased anisotropy inherent in flattened Gaussians exacerbates overfitting in sparse-view scenarios, hindering accurate surface fitting and degrading novel view synthesis performance. In this paper, we propose \net{}, a method that reconstructs more accurate and detailed surfaces while preserving high-quality novel view rendering. Our key insight is to introduce Stereo Geometry-Texture Alignment, which bridges rendering quality and geometry estimation, thereby jointly enhancing both surface reconstruction and view synthesis. In addition, we present a Pseudo-Feature Enhanced Geometry Consistency that enforces multi-view geometric consistency by incorporating both training and unseen views, effectively mitigating overfitting caused by sparse supervision. Extensive experiments on the DTU, BlendedMVS, and Mip-NeRF360 datasets demonstrate that our method achieves the state-of-the-art performance.

Method: Proposes SLAM-AGS framework that jointly optimizes weakly supervised similarity objective on slide-negative patches and self-supervised contrastive objective on slide-positive patches, with Adaptive Gradient Surgery to handle conflicting task gradients and prevent model collapse.

Result: On bone-marrow cytology dataset with witness rates from 10% to 0.5%, SLAM-AGS improves bag-level F1-Score and Top 400 positive cell retrieval over other pretraining methods, with largest gains at low witness rates.

Conclusion: Resolving gradient interference enables stable pretraining and better performance on downstream tasks in computational cytology, especially under low witness rate conditions.

Abstract: Computational cytology faces two major challenges: i) instance-level labels are unreliable and prohibitively costly to obtain, ii) witness rates are extremely low. We propose SLAM-AGS, a Slide-Label-Aware Multitask pretraining framework that jointly optimizes (i) a weakly supervised similarity objective on slide-negative patches and (ii) a self-supervised contrastive objective on slide-positive patches, yielding stronger performance on downstream tasks. To stabilize learning, we apply Adaptive Gradient Surgery to tackle conflicting task gradients and prevent model collapse. We integrate the pretrained encoder into an attention-based Multiple Instance Learning aggregator for bag-level prediction and attention-guided retrieval of the most abnormal instances in a bag. On a publicly available bone-marrow cytology dataset, with simulated witness rates from 10% down to 0.5%, SLAM-AGS improves bag-level F1-Score and Top 400 positive cell retrieval over other pretraining methods, with the largest gains at low witness rates, showing that resolving gradient interference enables stable pretraining and better performance on downstream tasks. To facilitate reproducibility, we share our complete implementation and evaluation framework as open source: https://github.com/Ace95/SLAM-AGS.

Method: Three-stage framework: 1) nnU-Net-based network produces initial airway mask, 2) skeleton-based algorithm identifies discontinuities and proposes reconnections, 3) 1D convolutional classifier determines true anatomical branches vs false paths.

Result: Outperforms existing 3D U-Net-based approaches (Bronchinet, NaviAirway) on both voxel-level and topological metrics across two datasets (ATM'22 and AeroPath), producing more complete and anatomically consistent airway trees.

Conclusion: RepAir provides robust airway segmentation that maintains high accuracy while addressing the critical issue of disconnected components through anatomically informed topology correction.

Abstract: Accurate airway segmentation from chest computed tomography (CT) scans is essential for quantitative lung analysis, yet manual annotation is impractical and many automated U-Net-based methods yield disconnected components that hinder reliable biomarker extraction. We present RepAir, a three-stage framework for robust 3D airway segmentation that combines an nnU-Net-based network with anatomically informed topology correction. The segmentation network produces an initial airway mask, after which a skeleton-based algorithm identifies potential discontinuities and proposes reconnections. A 1D convolutional classifier then determines which candidate links correspond to true anatomical branches versus false or obstructed paths. We evaluate RepAir on two distinct datasets: ATM'22, comprising annotated CT scans from predominantly healthy subjects and AeroPath, encompassing annotated scans with severe airway pathology. Across both datasets, RepAir outperforms existing 3D U-Net-based approaches such as Bronchinet and NaviAirway on both voxel-level and topological metrics, and produces more complete and anatomically consistent airway trees while maintaining high segmentation accuracy.

Method: HyMAD integrates spectral features from SincNet with temporal dependencies from RNNs, uses self-attention layers to strengthen intra-modal representations, and employs cross-modal fusion for robust multi-label classification of seismic events.

Result: The method achieves competitive performance on real-world field recordings and demonstrates generalization to complex simultaneous activity scenarios involving humans, animals, and vehicles.

Conclusion: HyMAD provides a modular framework for extending seismic-based activity recognition in real-world security applications, effectively addressing the challenge of detecting simultaneous activities in border surveillance.

Abstract: Seismic sensing has emerged as a promising solution for border surveillance and monitoring; the seismic sensors that are often buried underground are small and cannot be noticed easily, making them difficult for intruders to detect, avoid, or vandalize. This significantly enhances their effectiveness compared to highly visible cameras or fences. However, accurately detecting and distinguishing between overlapping activities that are happening simultaneously, such as human intrusions, animal movements, and vehicle rumbling, remains a major challenge due to the complex and noisy nature of seismic signals. Correctly identifying simultaneous activities is critical because failing to separate them can lead to misclassification, missed detections, and an incomplete understanding of the situation, thereby reducing the reliability of surveillance systems. To tackle this problem, we propose HyMAD (Hybrid Multi-Activity Detection), a deep neural architecture based on spatio-temporal feature fusion. The framework integrates spectral features extracted with SincNet and temporal dependencies modeled by a recurrent neural network (RNN). In addition, HyMAD employs self-attention layers to strengthen intra-modal representations and a cross-modal fusion module to achieve robust multi-label classification of seismic events. e evaluate our approach on a dataset constructed from real-world field recordings collected in the context of border surveillance and monitoring, demonstrating its ability to generalize to complex, simultaneous activity scenarios involving humans, animals, and vehicles. Our method achieves competitive performance and offers a modular framework for extending seismic-based activity recognition in real-world security applications.

Method: Proposed SemCoBench benchmark cleans and reorganizes video datasets based on action labels and subject-object pairs. SemCoFormer method uses Relationship Augmented Module (RAM) to distinguish similar relationships and Coherence Reasoning Module (CRM) to focus on relationship dynamics.

Result: Experimental results on SemCoBench show that modeling semantic coherence enables reasonable, fine-grained, and diverse visual relationship forecasting, improving video scene understanding.

Conclusion: Modeling semantic coherence is crucial for effective visual relationship forecasting, leading to more comprehensive video scene understanding through the proposed benchmark and transformer-based method.

Abstract: Visual Relationship Forecasting (VRF) aims to anticipate relations among objects without observing future visual content. The task relies on capturing and modeling the semantic coherence in object interactions, as it underpins the evolution of events and scenes in videos. However, existing VRF datasets offer limited support for learning such coherence due to noisy annotations in the datasets and weak correlations between different actions and relationship transitions in subject-object pair. Furthermore, existing methods struggle to distinguish similar relationships and overfit to unchanging relationships in consecutive frames. To address these challenges, we present SemCoBench, a benchmark that emphasizes semantic coherence for visual relationship forecasting. Based on action labels and short-term subject-object pairs, SemCoBench decomposes relationship categories and dynamics by cleaning and reorganizing video datasets to ensure predicting semantic coherence in object interactions. In addition, we also present Semantic Coherent Transformer method (SemCoFormer) to model the semantic coherence with a Relationship Augmented Module (RAM) and a Coherence Reasoning Module (CRM). RAM is designed to distinguish similar relationships, and CRM facilitates the model’s focus on the dynamics in relationships. The experimental results on SemCoBench demonstrate that modeling the semantic coherence is a key step toward reasonable, fine-grained, and diverse visual relationship forecasting, contributing to a more comprehensive understanding of video scenes.

Method: Inward sliding window attention mechanism with dual-path pipeline using cross-attention override strategy and cross-attention caching for efficiency.

Result: Generates ultra-high-resolution videos with fine-grained visual details and high efficiency, achieving superior performance on VBench compared to training-based alternatives.

Conclusion: The proposed training-free approach successfully enables high-resolution video generation while maintaining visual fidelity and global coherence with competitive efficiency.

Abstract: The quadratic time and memory complexity of the attention mechanism in modern Transformer based video generators makes end-to-end training for ultra high resolution videos prohibitively expensive. Motivated by this limitation, we introduce a training-free approach that leverages video Diffusion Transformers pretrained at their native scale to synthesize higher resolution videos without any additional training or adaptation. At the core of our method lies an inward sliding window attention mechanism, which originates from a key observation: maintaining each query token’s training scale receptive field is crucial for preserving visual fidelity and detail. However, naive local window attention, unfortunately, often leads to repetitive content and exhibits a lack of global coherence in the generated results. To overcome this challenge, we devise a dual-path pipeline that backs up window attention with a novel cross-attention override strategy, enabling the semantic content produced by local attention to be guided by another branch with a full receptive field and, therefore, ensuring holistic consistency. Furthermore, to improve efficiency, we incorporate a cross-attention caching strategy for this branch to avoid the frequent computation of full 3D attention. Extensive experiments demonstrate that our method delivers ultra-high-resolution videos with fine-grained visual details and high efficiency in a training-free paradigm. Meanwhile, it achieves superior performance on VBench, even compared to training-based alternatives, with competitive or improved efficiency. Codes are available at: https://github.com/WillWu111/FreeSwim

Method: Identifies latent collapse as the failure mode in naive joint training, draws analogy between diffusion and self-distillation, and proposes DSD framework with modified training objectives to stabilize latent space learning.

Result: Achieves FID=13.44/6.38/4.25 on ImageNet 256×256 with only 42M/118M/205M parameters and 50 training epochs, without classifier-free-guidance.

Conclusion: DSD enables stable end-to-end training of a single network that simultaneously learns encoding, decoding, and diffusion, outperforming traditional modular architectures.

Abstract: Standard Latent Diffusion Models rely on a complex, three-part architecture consisting of a separate encoder, decoder, and diffusion network, which are trained in multiple stages. This modular design is computationally inefficient, leads to suboptimal performance, and prevents the unification of diffusion with the single-network architectures common in vision foundation models. Our goal is to unify these three components into a single, end-to-end trainable network. We first demonstrate that a naive joint training approach fails catastrophically due to ``latent collapse’’, where the diffusion training objective interferes with the network’s ability to learn a good latent representation. We identify the root causes of this instability by drawing a novel analogy between diffusion and self-distillation based unsupervised learning method. Based on this insight, we propose Diffusion as Self-Distillation (DSD), a new framework with key modifications to the training objective that stabilize the latent space. This approach enables, for the first time, the stable end-to-end training of a single network that simultaneously learns to encode, decode, and perform diffusion. DSD achieves outstanding performance on the ImageNet $256\times 256$ conditional generation task: FID=13.44/6.38/4.25 with only 42M/118M/205M parameters and 50 training epochs on ImageNet, without using classifier-free-guidance.

Method: Propose a view-based approach with vector fields encoding views, using neural field-based implicit representation of 3D environments to support both direct queries (view assessment indices) and inverse queries (occlusion avoidance and pattern matching).

Result: Validated through quantitative experiments, real-world case studies, and expert feedback; effective for finding desirable viewpoints, analyzing building facade visibility, and evaluating outdoor space views.

Conclusion: The approach successfully enables efficient 3D urban data exploration for key analysis tasks like visibility assessments, solar exposure evaluation, and visual impact analysis of developments.

Abstract: Despite the growing availability of 3D urban datasets, extracting insights remains challenging due to computational bottlenecks and the complexity of interacting with data. In fact, the intricate geometry of 3D urban environments results in high degrees of occlusion and requires extensive manual viewpoint adjustments that make large-scale exploration inefficient. To address this, we propose a view-based approach for 3D data exploration, where a vector field encodes views from the environment. To support this approach, we introduce a neural field-based method that constructs an efficient implicit representation of 3D environments. This representation enables both faster direct queries, which consist of the computation of view assessment indices, and inverse queries, which help avoid occlusion and facilitate the search for views that match desired data patterns. Our approach supports key urban analysis tasks such as visibility assessments, solar exposure evaluation, and assessing the visual impact of new developments. We validate our method through quantitative experiments, case studies informed by real-world urban challenges, and feedback from domain experts. Results show its effectiveness in finding desirable viewpoints, analyzing building facade visibility, and evaluating views from outdoor spaces. Code and data are publicly available at https://urbantk.org/neural-3d.

Method: Uses VLMs to refine annotations via questionnaires extracting behavioral cues, splits data into high/low-reliability subsets, and applies curriculum learning with soft label refinement to gradually incorporate ambiguous samples.

Result: Outperforms prior state-of-the-art on engagement benchmarks: EngageNet (3/6 feature settings, max +1.21% improvement), DREAMS (+0.22 F1), and PAFE (+0.06 F1).

Conclusion: Addressing label subjectivity with VLMs and curriculum learning improves engagement recognition performance, demonstrating benefits of refined supervision for noisy video datasets.

Abstract: Engagement recognition in video datasets, unlike traditional image classification tasks, is particularly challenged by subjective labels and noise limiting model performance. To overcome the challenges of subjective and noisy engagement labels, we propose a framework leveraging Vision Large Language Models (VLMs) to refine annotations and guide the training process. Our framework uses a questionnaire to extract behavioral cues and split data into high- and low-reliability subsets. We also introduce a training strategy combining curriculum learning with soft label refinement, gradually incorporating ambiguous samples while adjusting supervision to reflect uncertainty. We demonstrate that classical computer vision models trained on refined high-reliability subsets and enhanced with our curriculum strategy show improvements, highlighting benefits of addressing label subjectivity with VLMs. This method surpasses prior state of the art across engagement benchmarks such as EngageNet (three of six feature settings, maximum improvement of +1.21%), and DREAMS / PAFE with F1 gains of +0.22 / +0.06.

Method: Distills a light-weight confidence predictor to rank tokens by uncertainty and selectively merges low-confidence tokens, using confidence signals to identify regions emphasized by the transformer.

Result: Achieves up to 11.3x speedup on VGGT and 7.2x speedup on MapAnything, with speedups scaling with sequence length and no performance degradation.

Conclusion: Co-Me enables substantial acceleration of visual geometric transformers for real-time applications while maintaining spatial coverage and model performance.

Abstract: We propose Confidence-Guided Token Merging (Co-Me), an acceleration mechanism for visual geometric transformers without retraining or finetuning the base model. Co-Me distilled a light-weight confidence predictor to rank tokens by uncertainty and selectively merge low-confidence ones, effectively reducing computation while maintaining spatial coverage. Compared to similarity-based merging or pruning, the confidence signal in Co-Me reliably indicates regions emphasized by the transformer, enabling substantial acceleration without degrading performance. Co-Me applies seamlessly to various multi-view and streaming visual geometric transformers, achieving speedups that scale with sequence length. When applied to VGGT and MapAnything, Co-Me achieves up to $11.3\times$ and $7.2\times$ speedup, making visual geometric transformers practical for real-time 3D perception and reconstruction.

Method: Enhanced model architecture and training pipeline; unified RL strategy with shared reward models for joint image generation/editing improvement; light Edit Instruction Alignment stage for better editing comprehension.

Result: Achieves 0.89 on GenEval and 4.31 on ImgEdit benchmarks, surpassing state-of-the-art models like BAGEL and approaching proprietary model performance.

Conclusion: UniGen-1.5 demonstrates competitive multimodal capabilities through unified RL training and instruction alignment, enabling strong performance across image understanding, generation, and editing tasks.

Abstract: We present UniGen-1.5, a unified multimodal large language model (MLLM) for advanced image understanding, generation and editing. Building upon UniGen, we comprehensively enhance the model architecture and training pipeline to strengthen the image understanding and generation capabilities while unlocking strong image editing ability. Especially, we propose a unified Reinforcement Learning (RL) strategy that improves both image generation and image editing jointly via shared reward models. To further enhance image editing performance, we propose a light Edit Instruction Alignment stage that significantly improves the editing instruction comprehension that is essential for the success of the RL training. Experimental results show that UniGen-1.5 demonstrates competitive understanding and generation performance. Specifically, UniGen-1.5 achieves 0.89 and 4.31 overall scores on GenEval and ImgEdit that surpass the state-of-the-art models such as BAGEL and reaching performance comparable to proprietary models such as GPT-Image-1.

Method: Proposes a Homological Parity Principle where even-dimensional homology represents Structure/Context (generative models) and odd-dimensional homology represents Flow/Content (sensory/memory data), governed by a Context-Content Uncertainty Principle that creates dynamical cycles between these parities.

Result: The framework distinguishes two cognitive modes: Inference (waking) as Context-before-Content process, and Learning (sleep) as Structure-before-Specificity process. It unifies semantic and episodic memory functions.

Conclusion: This topological approach provides a structural generalization of existing cognitive theories, recasting Friston’s Free Energy Principle and Tonini’s Integrated Information Theory in topological terms through the homological parity interpretation.

Abstract: We propose a formal foundation for cognition rooted in algebraic topology, built on a Homological Parity Principle. This posits that even-dimensional homology represents stable Structure/Context (e.g., generative models), while odd-dimensional homology represents dynamic Flow/Content (e.g., sensory/memory data). Cognition is governed by the Context-Content Uncertainty Principle (CCUP), a dynamical cycle aligning these parities. This framework distinguishes two modes: Inference (waking), where the scaffold predicts the flow (a Context-before-Content process); and Learning (sleep), an inverted Structure-before-Specificity process where memory traces sculpt the scaffold. This parity interpretation unifies cognitive functions like semantic and episodic memory and provides a structural generalization of existing theories, recasting Friston’s Free Energy Principle and Tonini’s Integrated Information in topological terms.

Method: Two-stage framework: 1) Pretraining with unified past-future self-reconstruction and velocity-based masking of highly dynamic joints; 2) Fine-tuning with full future sequence prediction and lightweight future-text prediction head for joint optimization of motion prediction and understanding.

Result: Reduces average prediction errors by 8.8% over state-of-the-art methods on Human3.6M, 3DPW, and AMASS datasets, while achieving competitive future-motion understanding performance compared to LLM-based models.

Conclusion: The proposed self-supervised framework effectively addresses representation shortcutting in motion prediction by decoupling representation learning from prediction, leading to more accurate and temporally consistent 3D human motion predictions.

Abstract: 3D human motion prediction aims to generate coherent future motions from observed sequences, yet existing end-to-end regression frameworks often fail to capture complex dynamics and tend to produce temporally inconsistent or static predictions-a limitation rooted in representation shortcutting, where models rely on superficial cues rather than learning meaningful motion structure. We propose a two-stage self-supervised framework that decouples representation learning from prediction. In the pretraining stage, the model performs unified past-future self-reconstruction, reconstructing the past sequence while recovering masked joints in the future sequence under full historical guidance. A velocity-based masking strategy selects highly dynamic joints, forcing the model to focus on informative motion components and internalize the statistical dependencies between past and future states without regression interference. In the fine-tuning stage, the pretrained model predicts the entire future sequence, now treated as fully masked, and is further equipped with a lightweight future-text prediction head for joint optimization of low-level motion prediction and high-level motion understanding. Experiments on Human3.6M, 3DPW, and AMASS show that our method reduces average prediction errors by 8.8% over state-of-the-art methods while achieving competitive future-motion understanding performance compared to LLM-based models. Code is available at: https://github.com/JunyuShi02/MoReFun

Method: AGPNet uses a dual-branch architecture: a reconstruction branch that learns to reconstruct normal samples, and an auxiliary attention branch that produces sample-aware attention masks to guide noise perturbation. This allows more aggressive perturbation at important regions to encourage learning of invariant normal patterns.

Result: Extensive experiments on MVTec-AD, VisA, and MVTec-3D benchmarks show that AGPNet consistently achieves leading anomaly detection performance across various setups including few-shot, one-class, and multi-class scenarios.

Conclusion: Attention-guided perturbation effectively improves reconstruction-based anomaly detection by focusing on important regions, leading to more robust learning of normal patterns and better anomaly detection performance across multiple benchmarks and setups.

Abstract: In unsupervised image anomaly detection, reconstruction methods aim to train models to capture normal patterns comprehensively for normal data reconstruction. Yet, these models sometimes retain unintended reconstruction capacity for anomalous regions during inference, leading to missed detections. To mitigate this issue, existing works perturb normal samples in a sample-agnostic manner, uniformly adding noise across spatial locations before reconstructing the original. Despite promising results, they disregard the fact that foreground locations are inherently more critical for robust reconstruction. Motivated by this, we present a novel reconstruction framework named Attention-Guided Perturbation Network (AGPNet) for industrial anomaly detection. Its core idea is to add perturbations guided by a sample-aware attention mask to improve the learning of invariant normal patterns at important locations. AGPNet consists of two branches, \ie, a reconstruction branch and an auxiliary attention-based perturbation one. The reconstruction branch learns to reconstruct normal samples, while the auxiliary one aims to produce attention masks to guide the noise perturbation process for normal samples. By perturbing more aggressively at those important regions, we encourage the reconstruction branch to learn inherent normal patterns both comprehensively and robustly. Extensive experiments are conducted on several popular benchmarks covering MVTec-AD, VisA, and MVTec-3D, and show that AGPNet consistently obtains leading anomaly detection performance across a variety of setups, including few-shot, one-class, and multi-class ones.

Method: Introduces Light Enhanced Depth (LED), a cost-effective approach that uses high-definition headlights to project patterns for improved depth estimation in low-light environments.

Result: Significant performance improvements across multiple depth-estimation architectures on both synthetic and real datasets, with enhanced scene understanding beyond illuminated areas.

Conclusion: LED effectively improves depth estimation reliability in nighttime conditions, and the Nighttime Synthetic Drive Dataset (49,990 annotated images) is released to support further research.

Abstract: Nighttime camera-based depth estimation is a highly challenging task, especially for autonomous driving applications, where accurate depth perception is essential for ensuring safe navigation. Models trained on daytime data often fail in the absence of precise but costly LiDAR. Even vision foundation models trained on large amounts of data are unreliable in low-light conditions. In this work, we aim to improve the reliability of perception systems at night time. To this end, we introduce Light Enhanced Depth (LED), a novel, cost-effective approach that significantly improves depth estimation in low-light environments by harnessing a pattern projected by high definition headlights available in modern vehicles. LED leads to significant performance boosts across multiple depth-estimation architectures (encoder-decoder, Adabins, DepthFormer, Depth Anything V2) both on synthetic and real datasets. Furthermore, increased performances beyond illuminated areas reveal a holistic enhancement in scene understanding. Finally, we release the Nighttime Synthetic Drive Dataset, a synthetic and photo-realistic nighttime dataset, which comprises 49,990 comprehensively annotated images.

Method: Combines theoretical foundations with practical applications, reviewing state-of-the-art advancements including convolutional neural networks (CNNs), YOLO architectures, and transformer-based approaches like DETR.

Result: Provides comprehensive analysis of big data processing with emphasis on model optimization and performance evaluation metrics for object detection in complex environments.

Conclusion: Offers valuable insights for researchers, data scientists, and engineers by examining AI techniques and large language models integration for enhancing object detection capabilities.

Abstract: An in-depth exploration of object detection and semantic segmentation is provided, combining theoretical foundations with practical applications. State-of-the-art advancements in machine learning and deep learning are reviewed, focusing on convolutional neural networks (CNNs), YOLO architectures, and transformer-based approaches such as DETR. The integration of artificial intelligence (AI) techniques and large language models for enhancing object detection in complex environments is examined. Additionally, a comprehensive analysis of big data processing is presented, with emphasis on model optimization and performance evaluation metrics. By bridging the gap between traditional methods and modern deep learning frameworks, valuable insights are offered for researchers, data scientists, and engineers aiming to apply AI-driven methodologies to large-scale object detection tasks.

Method: Uses unified canonical projection (UCP) for consistent sensor features and cross-attention across sensors along patches (CASAP) for robustness against sensor issues.

Result: Achieves 9.7% improvement in AP BEV (87.2%) and 20.1% in AP 3D (73.6%) on K-Radar dataset, with low computational cost.

Conclusion: ASF provides superior object detection performance under various weather and sensor degradation conditions compared to state-of-the-art methods.

Abstract: Sensor fusion of camera, LiDAR, and 4-dimensional (4D) Radar has brought a significant performance improvement in autonomous driving. However, there still exist fundamental challenges: deeply coupled fusion methods assume continuous sensor availability, making them vulnerable to sensor degradation and failure, whereas sensor-wise cross-attention fusion methods struggle with computational cost and unified feature representation. This paper presents availability-aware sensor fusion (ASF), a novel method that employs unified canonical projection (UCP) to enable consistency in all sensor features for fusion and cross-attention across sensors along patches (CASAP) to enhance robustness of sensor fusion against sensor degradation and failure. As a result, the proposed ASF shows a superior object detection performance to the existing state-of-the-art fusion methods under various weather and sensor degradation (or failure) conditions. Extensive experiments on the K-Radar dataset demonstrate that ASF achieves improvements of 9.7% in AP BEV (87.2%) and 20.1% in AP 3D (73.6%) in object detection at IoU=0.5, while requiring a low computational cost. All codes are available at https://github.com/kaist-avelab/k-radar.

Method: Uses a three-pronged approach: (1) point-matching mask propagation using DDIM inversion features, (2) training-free AdaIN-guided localized stylization at latent and attention levels, (3) sliding-window consistent smoothing with optical flow and noise refinement.

Result: UniVST demonstrates superior performance to existing methods in both quantitative and qualitative metrics, effectively preserving primary object style while ensuring temporal consistency and detail preservation.

Conclusion: The proposed UniVST framework successfully addresses the challenges of localized video style transfer without training requirements, achieving better temporal consistency and detail preservation compared to existing methods.

Abstract: This paper presents UniVST, a unified framework for localized video style transfer based on diffusion models. It operates without the need for training, offering a distinct advantage over existing diffusion methods that transfer style across entire videos. The endeavors of this paper comprise: (1) A point-matching mask propagation strategy that leverages the feature maps from the DDIM inversion. This streamlines the model’s architecture by obviating the need for tracking models. (2) A training-free AdaIN-guided localized video stylization mechanism that operates at both the latent and attention levels. This balances content fidelity and style richness, mitigating the loss of localized details commonly associated with direct video stylization. (3) A sliding-window consistent smoothing scheme that harnesses optical flow within the pixel representation and refines predicted noise to update the latent space. This significantly enhances temporal consistency and diminishes artifacts in stylized video. Our proposed UniVST has been validated to be superior to existing methods in quantitative and qualitative metrics. It adeptly addresses the challenges of preserving the primary object’s style while ensuring temporal consistency and detail preservation. Our code is available at https://github.com/QuanjianSong/UniVST.

Method: Uses semantic contrastive alignment loss for parameter-free modality alignment and synchronized alignment fusion with hierarchical filtering in Fourier domain for efficient bi-modal feature fusion.

Result: Reduces parameters by 70.0%, FLOPs by 49.4%, increases inference speed by 152.5% compared to state-of-the-art MROS, while achieving better performance across most metrics on multiple datasets.

Conclusion: AlignSal enables efficient real-time bi-modal salient object detection suitable for AAV devices, demonstrating superior performance and generalizability compared to 19 state-of-the-art models.

Abstract: Autonomous aerial vehicle (AAV)-based bi-modal salient object detection (BSOD) aims to segment salient objects in a scene utilizing complementary cues in unaligned RGB and thermal image pairs. However, the high computational expense of existing AAV-based BSOD models limits their applicability to real-world AAV devices. To address this problem, we propose an efficient Fourier filter network with contrastive learning that achieves both real-time and accurate performance. Specifically, we first design a semantic contrastive alignment loss to align the two modalities at the semantic level, which facilitates mutual refinement in a parameter-free way. Second, inspired by the fast Fourier transform that obtains global relevance in linear complexity, we propose synchronized alignment fusion, which aligns and fuses bi-modal features in the channel and spatial dimensions by a hierarchical filtering mechanism. Our proposed model, AlignSal, reduces the number of parameters by 70.0%, decreases the floating point operations by 49.4%, and increases the inference speed by 152.5% compared to the cutting-edge BSOD model (i.e., MROS). Extensive experiments on the AAV RGB-T 2400 and seven bi-modal dense prediction datasets demonstrate that AlignSal achieves both real-time inference speed and better performance and generalizability compared to nineteen state-of-the-art models across most evaluation metrics. In addition, our ablation studies further verify AlignSal’s potential in boosting the performance of existing aligned BSOD models on AAV-based unaligned data. The code is available at: https://github.com/JoshuaLPF/AlignSal.

Method: Uses foundation image tagging model to capture visual features in natural language, LLM to select target-class features, translates potential biases to vision-language embeddings, and implements in-processing bias mitigation.

Result: Outperforms state-of-the-art methods on diverse datasets (CelebA, Waterbirds, ImageNet, UrbanCars) by effectively detecting and mitigating a wide range of biases.

Conclusion: MAVias provides a comprehensive solution for open-set bias mitigation in visual recognition tasks by leveraging foundation models to handle multiple unknown biases.

Abstract: Mitigating biases in computer vision models is an essential step towards the trustworthiness of artificial intelligence models. Existing bias mitigation methods focus on a small set of predefined biases, limiting their applicability in visual datasets where multiple, possibly unknown biases exist. To address this limitation, we introduce MAVias, an open-set bias mitigation approach leveraging foundation models to discover spurious associations between visual attributes and target classes. MAVias first captures a wide variety of visual features in natural language via a foundation image tagging model, and then leverages a large language model to select those visual features defining the target class, resulting in a set of language-coded potential visual biases. We then translate this set of potential biases into vision-language embeddings and introduce an in-processing bias mitigation approach to prevent the model from encoding information related to them. Our experiments on diverse datasets, including CelebA, Waterbirds, ImageNet, and UrbanCars, show that MAVias effectively detects and mitigates a wide range of biases in visual recognition tasks outperforming current state-of-the-art.

Method: Developed AdCare-VLM based on LLaVA with unified visual latent space pre-alignment. Used 806 custom-annotated TB medication videos to fine-tune for adherence pattern detection. Created LLM-TB-VQA dataset with positive, negative, and ambiguous adherence cases.

Result: Outperformed parameter-efficient fine-tuning enabled VLM models (LLaVA-V1.5, Chat-UniVi) with absolute improvements of 3.1-3.54% across different configurations. Identified correlations between visual features and medical concepts.

Conclusion: The proposed method effectively integrates visual-linguistic representations for medication adherence monitoring, showing superior performance and enhanced interpretability through attention map visualizations.

Abstract: Chronic diseases, including diabetes, hypertension, asthma, HIV-AIDS, epilepsy, and tuberculosis, necessitate rigorous adherence to medication to avert disease progression, manage symptoms, and decrease mortality rates. Adherence is frequently undermined by factors including patient behavior, caregiver support, elevated medical costs, and insufficient healthcare infrastructure. We propose AdCare-VLM, a specialized LLaVA-based multimodal large vision language model (LVLM) by introducing a unified visual latent space with pre-alignment to facilitate visual question answering (VQA) concerning medication adherence through patient videos. We employ a private dataset comprising 806 custom-annotated tuberculosis (TB) medication monitoring videos, which have been labeled by clinical experts, to fine-tune the model for adherence pattern detection. We present LLM-TB-VQA, a detailed medical adherence VQA dataset that encompasses positive, negative, and ambiguous adherence cases. Our method identifies correlations between visual features, such as the clear visibility of the patient’s face, medication, water intake, and the act of ingestion, and their associated medical concepts in captions. This facilitates the integration of aligned visual-linguistic representations and improves multimodal interactions. Experimental results indicate that our method surpasses parameter-efficient fine-tuning (PEFT) enabled VLM models, such as LLaVA-V1.5 and Chat-UniVi, with absolute improvements ranging from 3.1% to 3.54% across pre-trained, regular, and low-rank adaptation (LoRA) configurations. Comprehensive ablation studies and attention map visualizations substantiate our approach, enhancing interpretability.

Method: Uses learned motion queries parallel to detection/tracking queries with shared reference points, plus interactive semantic decoding for better information exchange.

Result: Experiments on nuScenes dataset with UniAD and SparseDrive show performance improvements across perception, prediction, and planning tasks.

Conclusion: The divide and merge approach effectively addresses negative transfer by separating semantic and motion learning while maintaining unified reference points.

Abstract: Perceiving the environment and its changes over time corresponds to two fundamental yet heterogeneous types of information: semantics and motion. Previous end-to-end autonomous driving works represent both types of information in a single feature vector. However, including motion related tasks, such as prediction and planning, impairs detection and tracking performance, a phenomenon known as negative transfer in multi-task learning. To address this issue, we propose Neural-Bayes motion decoding, a novel parallel detection, tracking, and prediction method that separates semantic and motion learning. Specifically, we employ a set of learned motion queries that operate in parallel with detection and tracking queries, sharing a unified set of recursively updated reference points. Moreover, we employ interactive semantic decoding to enhance information exchange in semantic tasks, promoting positive transfer. Experiments on the nuScenes dataset with UniAD and SparseDrive confirm the effectiveness of our divide and merge approach, resulting in performance improvements across perception, prediction, and planning. Our code is available at https://github.com/shenyinzhe/DMAD.

Method: Investigated various denoising-based generative models without noise conditioning, provided theoretical analysis of the error from removing noise conditioning, and introduced a noise-unconditional model.

Result: Most models showed graceful degradation without noise conditioning, sometimes performing better. The noise-unconditional model achieved FID of 2.23 on CIFAR-10, narrowing the gap to leading noise-conditional models.

Conclusion: The findings suggest noise conditioning may not be essential for denoising generative models, encouraging the community to revisit their foundations and formulations.

Abstract: It is widely believed that noise conditioning is indispensable for denoising diffusion models to work successfully. This work challenges this belief. Motivated by research on blind image denoising, we investigate a variety of denoising-based generative models in the absence of noise conditioning. To our surprise, most models exhibit graceful degradation, and in some cases, they even perform better without noise conditioning. We provide a theoretical analysis of the error caused by removing noise conditioning and demonstrate that our analysis aligns with empirical observations. We further introduce a noise-unconditional model that achieves a competitive FID of 2.23 on CIFAR-10, significantly narrowing the gap to leading noise-conditional models. We hope our findings will inspire the community to revisit the foundations and formulations of denoising generative models.

Method: Approximates dataset topology by constructing adjacency graphs using Uniform Manifold Approximation and Projection (UMAP) to capture nonlinear correlations in hyperspectral images from X-ray transmission spectroscopy.

Result: Significantly improves machine learning algorithm performance, preserves global data structure, and enhances feature separability for more accurate classification.

Conclusion: UMAP-based topology approximation effectively addresses limitations of traditional feature extraction methods for XRT multi-energy images, leading to more robust and accurate classification results.

Abstract: Traditional feature extraction and projection techniques, such as Principal Component Analysis, struggle to adequately represent X-Ray Transmission (XRT) Multi-Energy (ME) images, limiting the performance of neural networks in decision-making processes. To address this issue, we propose a method that approximates the dataset topology by constructing adjacency graphs using the Uniform Manifold Approximation and Projection. This approach captures nonlinear correlations within the data, significantly improving the performance of machine learning algorithms, particularly in processing Hyperspectral Images (HSI) from X-ray transmission spectroscopy. This technique not only preserves the global structure of the data but also enhances feature separability, leading to more accurate and robust classification results.

Method: Integrates pre-trained detector, pre-trained segmentor with tracking logic into a zero-shot MOT system that requires no fine-tuning, making segmentation the central component rather than auxiliary information.

Result: Achieves state-of-the-art results on DanceTrack (+2.1 HOTA and +4.5 IDF1), UAVDT, and BDD100K benchmarks, demonstrating superior performance in handling challenging tracking scenarios.

Conclusion: SAM2MOT significantly reduces dependence on labeled data and paves the way for transitioning MOT research from task-specific solutions to general-purpose systems through its segmentation-driven approach.

Abstract: Inspired by Segment Anything 2, which generalizes segmentation from images to videos, we propose SAM2MOT–a novel segmentation-driven paradigm for multi-object tracking that breaks away from the conventional detection-association framework. In contrast to previous approaches that treat segmentation as auxiliary information, SAM2MOT places it at the heart of the tracking process, systematically tackling challenges like false positives and occlusions. Its effectiveness has been thoroughly validated on major MOT benchmarks. Furthermore, SAM2MOT integrates pre-trained detector, pre-trained segmentor with tracking logic into a zero-shot MOT system that requires no fine-tuning. This significantly reduces dependence on labeled data and paves the way for transitioning MOT research from task-specific solutions to general-purpose systems. Experiments on DanceTrack, UAVDT, and BDD100K show state-of-the-art results. Notably, SAM2MOT outperforms existing methods on DanceTrack by +2.1 HOTA and +4.5 IDF1, highlighting its effectiveness in MOT. Code is available at https://github.com/TripleJoy/SAM2MOT.

Method: Developed StructuralGeo simulation engine to generate synthetic 3D geological models, then trained conditional generative flow-matching models with 3D attention U-net architecture using this data.

Result: The foundation model can reconstruct multiple plausible 3D geological scenarios from surface topography and sparse borehole data, depicting structures like layers, faults, folds, and dikes.

Conclusion: The combination of geological simulation and generative AI provides a flexible prior for probabilistic modeling, regional fine-tuning, and use as an AI-based regularizer in traditional geophysical inversion workflows.

Abstract: Reconstructing the structural geology and mineral composition of the first few kilometers of the Earth’s subsurface from sparse or indirect surface observations remains a long-standing challenge with critical applications in mineral exploration, geohazard assessment, and geotechnical engineering. This inherently ill-posed problem is often addressed by classical geophysical inversion methods, which typically yield a single maximum-likelihood model that fails to capture the full range of plausible geology. The adoption of modern deep learning methods has been limited by the lack of large 3D training datasets. We address this gap with \textit{StructuralGeo}, a geological simulation engine that mimics eons of tectonic, magmatic, and sedimentary processes to generate a virtually limitless supply of realistic synthetic 3D lithological models. Using this dataset, we train both unconditional and conditional generative flow-matching models with a 3D attention U-net architecture. The resulting foundation model can reconstruct multiple plausible 3D scenarios from surface topography and sparse borehole data, depicting structures such as layers, faults, folds, and dikes. By sampling many reconstructions from the same observations, we introduce a probabilistic framework for estimating the size and extent of subsurface features. While the realism of the output is bounded by the fidelity of the training data to true geology, this combination of simulation and generative AI functions offers a flexible prior for probabilistic modeling, regional fine-tuning, and use as an AI-based regularizer in traditional geophysical inversion workflows.

Method: The authors collected 711 train driver performance measurements from controlled experiments, measuring reaction time and distance to obstacles under various conditions including different speeds, obstacle sizes, train protection systems, and obstacle color contrasts.

Result: A new public and anonymized dataset was created and published at https://data.fid-move.de/de/dataset/atosensedata, providing comprehensive measurements for research, standardization and regulatory purposes.

Conclusion: This dataset helps remedy the lack of published performance data for obstacle detection in driverless train systems, supporting unbiased research, standardization efforts, and regulatory compliance under EU regulations.

Abstract: Points 2.1.4(b), 2.4.2(b) and 2.4.3(b) in Annex I of Implementing Regulation (EU) No. 402/2013 allow a simplified approach for the safety approval of computer vision systems for driverless trains, if they have ‘similar’ functions and interfaces as the replaced human driver. The human driver is not replaced one-to-one by a technical system - only a limited set of cognitive functions are replaced. However, performance in the most challenging function, obstacle detection, is difficult to quantify due to the deficiency of published measurement results. This article summarizes the data published so far. This article also goes a long way to remedy this situation by providing a new public and anonymized dataset of 711 train driver performance measurements from controlled experiments. The measurements are made for different speeds, obstacle sizes, train protection systems and obstacle color contrasts respectively. The measured values are reaction time and distance to the obstacle. The goal of this paper is an unbiased and exhaustive description of the presented dataset for research, standardization and regulation. The dataset with supplementing information and literature is published on https://data.fid-move.de/de/dataset/atosensedata

Method: Uses frequency filtering curriculum (low-frequency first) and Gaussian noise patching augmentation during pre-training of ViT-B/16 backbone on ImageNet-1K.

Result: Achieves 1.6x faster pre-training time and 2.25x fewer FLOPs while maintaining matching robustness on ImageNet-C and competitive linear probing performance compared to baseline.

Conclusion: The method provides dual benefits of efficiency and robustness, making large-scale self-supervised foundation modeling more accessible and opening new exploration avenues for data curriculum and augmentation.

Abstract: Large-scale vision foundation models such as DINOv2 boast impressive performances by leveraging massive architectures and training datasets. But numerous scenarios require practitioners to reproduce those pre-training solutions, such as on private data, new modalities, or simply for scientific questioning–which is currently extremely demanding computation-wise. We thus propose a novel pre-training strategy for DINOv2 that simultaneously accelerates convergence–and strengthens robustness to common corruptions as a by-product. Our approach involves a frequency filtering curriculum–low-frequency being seen first–and the Gaussian noise patching augmentation. Applied to a ViT-B/16 backbone trained on ImageNet-1K, while pre-training time and FLOPs are reduced by 1.6x and 2.25x, our method still achieves matching robustness in corruption benchmarks (ImageNet-C) and maintains competitive linear probing performance compared with baseline. This dual benefit of efficiency and robustness makes large-scale self-supervised foundation modeling more attainable, while opening the door to novel exploration around data curriculum and augmentation as means to improve self-supervised learning models robustness. The code is available at https://github.com/KevinZ0217/fast_dinov2

Method: The research uses a convolutional neural network (CNN) model for rice grain classification, evaluated using performance metrics including accuracy, recall, precision, F1-Score, ROC curves, and confusion matrix analysis. Explainability techniques LIME and SHAP are integrated to understand model decisions.

Result: The CNN model achieved remarkable accuracy with perfect area under ROC curves for each class. Confusion matrix analysis showed minimal misclassifications, confirming the model’s effectiveness in distinguishing between different rice varieties.

Conclusion: The proposed CNN-based framework provides an effective and efficient automated solution for rice grain classification, with explainability techniques offering valuable insights into the model’s decision-making process for different rice grain features.

Abstract: Rice is an essential staple food worldwide that is important in promoting international trade, economic growth, and nutrition. Asian countries such as China, India, Pakistan, Thailand, Vietnam, and Indonesia are notable for their significant contribution to the cultivation and utilization of rice. These nations are also known for cultivating different rice grains, including short and long grains. These sizes are further classified as basmati, jasmine, kainat saila, ipsala, arborio, etc., catering to diverse culinary preferences and cultural traditions. For both local and international trade, inspecting and maintaining the quality of rice grains to satisfy customers and preserve a country’s reputation is necessary. Manual quality check and classification is quite a laborious and time-consuming process. It is also highly prone to mistakes. Therefore, an automatic solution must be proposed for the effective and efficient classification of different varieties of rice grains. This research paper presents an automatic framework based on a convolutional neural network (CNN) for classifying different varieties of rice grains. We evaluated the proposed model based on performance metrics such as accuracy, recall, precision, and F1-Score. The CNN model underwent rigorous training and validation, achieving a remarkable accuracy rate and a perfect area under each class’s Receiver Operating Characteristic (ROC) curve. The confusion matrix analysis confirmed the model’s effectiveness in distinguishing between the different rice varieties, indicating minimal misclassifications. Additionally, the integration of explainability techniques such as LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations) provided valuable insights into the model’s decision-making process, revealing how specific features of the rice grains influenced classification outcomes.

Method: Introduces coalescent projection to replace soft prompts and pseudo-class generation method using self-supervised transformations from base domain only.

Result: Outperforms latest SOTA methods on extreme domain-shift problem of BSCD-FSL benchmark.

Conclusion: The proposed method effectively addresses overfitting in cross-domain few-shot learning and demonstrates superior performance on challenging domain-shift scenarios.

Abstract: Despite the progress in cross-domain few-shot learning, a model pre-trained with DINO combined with a prototypical classifier outperforms the latest SOTA methods. A crucial limitation that needs to be overcome is that updating too many parameters of the transformers leads to overfitting due to the scarcity of labeled samples. To address this challenge, we propose a new concept, coalescent projection, as an effective successor to soft prompts. Additionally, we propose a novel pseudo-class generation method, combined with self-supervised transformations, that relies solely on the base domain to prepare the network to encounter unseen samples from different domains. The proposed method exhibits its effectiveness in comprehensive experiments on the extreme domain-shift problem of the BSCD-FSL benchmark. Our code is published at \href{https://github.com/Naeem-Paeedeh/CPLSR}{https://github.com/Naeem-Paeedeh/CPLSR}.

Method: Created Logos dataset - the largest RSL dataset by number of signers, with explicit annotation of visually similar sign groups. Used transfer learning approaches including joint training with multiple classification heads and few-shot learning.

Result: Pre-trained models on Logos can serve as universal encoders for other language SLR tasks. Explicit labeling of visually similar signs improves model quality. Achieved state-of-the-art results on WLASL dataset and competitive results on AUTSL dataset with single RGB stream model.

Conclusion: The Logos dataset enables effective cross-language transfer learning and explicit annotation of visually similar signs enhances model performance, making it a valuable resource for sign language recognition research.

Abstract: This paper examines two aspects of the isolated sign language recognition (ISLR) task. First, although a certain number of datasets is available, the data for individual sign languages is limited. It poses the challenge of cross-language ISLR model training, including transfer learning. Second, similar signs can have different semantic meanings. It leads to ambiguity in dataset labeling and raises the question of the best policy for annotating such signs. To address these issues, this study presents Logos, a novel Russian Sign Language (RSL) dataset, the most extensive available ISLR dataset by the number of signers, one of the most extensive datasets in size and vocabulary, and the largest RSL dataset. It is shown that a model, pre-trained on the Logos dataset can be used as a universal encoder for other language SLR tasks, including few-shot learning. We explore cross-language transfer learning approaches and find that joint training using multiple classification heads benefits accuracy for the target low-resource datasets the most. The key feature of the Logos dataset is explicitly annotated visually similar sign groups. We show that explicitly labeling visually similar signs improves trained model quality as a visual encoder for downstream tasks. Based on the proposed contributions, we outperform current state-of-the-art results for the WLASL dataset and get competitive results for the AUTSL dataset, with a single stream model processing solely RGB video. The source code, dataset, and pre-trained models are publicly available.

Method: Uses a multi-agent system with specialized agents (morphology, spatial context) drawing from pathology textbooks to generate diverse clinical descriptions, and encodes text as a list of descriptions rather than single prompts.

Result: Shows improved performance over single-prompt baselines and achieves results comparable to state-of-the-art models on renal and lung cancer datasets.

Conclusion: The proposed framework effectively addresses VLM limitations in pathology MIL by generating grounded, diverse clinical descriptions and using list-based encoding for better text-visual alignment.

Abstract: Multiple Instance Learning (MIL) is the leading approach for whole slide image (WSI) classification, enabling efficient analysis of gigapixel pathology slides. Recent work has introduced vision-language models (VLMs) into MIL pipelines to incorporate medical knowledge through text-based class descriptions rather than simple class names. However, when these methods rely on large language models (LLMs) to generate clinical descriptions or use fixed-length prompts to represent complex pathology concepts, the limited token capacity of VLMs often constrains the expressiveness and richness of the encoded class information. Additionally, descriptions generated solely by LLMs may lack domain grounding and fine-grained medical specificity, leading to suboptimal alignment with visual features. To address these challenges, we propose a vision-language MIL framework with two key contributions: (1) A grounded multi-agent description generation system that leverages curated pathology textbooks and agent specialization (e.g., morphology, spatial context) to produce accurate and diverse clinical descriptions; (2) A text encoding strategy using a list of descriptions rather than a single prompt, capturing fine-grained and complementary clinical signals for better alignment with visual features. Integrated into a VLM-MIL pipeline, our approach shows improved performance over single-prompt class baselines and achieves results comparable to state-of-the-art models, as demonstrated on renal and lung cancer datasets.

Method: Proposes VidCom2 framework that quantifies frame uniqueness and adaptively adjusts compression intensity across frames to preserve essential information while reducing redundancy.

Result: Achieves 99.6% of original performance on LLaVA-OV with only 25% visual tokens, reduces 70.8% LLM generation latency, and is compatible with other token compression methods.

Conclusion: VidCom2 provides superior performance and efficiency for VideoLLMs through adaptive frame compression, effectively balancing token reduction with information preservation.

Abstract: Video large language models (VideoLLM) excel at video understanding, but face efficiency challenges due to the quadratic complexity of abundant visual tokens. Our systematic analysis of token compression methods for VideoLLMs reveals two critical issues: (i) overlooking distinctive visual signals across frames, leading to information loss; (ii) suffering from implementation constraints, causing incompatibility with modern architectures or efficient operators. To address these challenges, we distill three design principles for VideoLLM token compression and propose a plug-and-play inference acceleration framework “Video Compression Commander” (VidCom2). By quantifying each frame’s uniqueness, VidCom2 adaptively adjusts compression intensity across frames, effectively preserving essential information while reducing redundancy in video sequences. Extensive experiments across various VideoLLMs and benchmarks demonstrate the superior performance and efficiency of our VidCom2. With only 25% visual tokens, VidCom2 achieves 99.6% of the original performance on LLaVA-OV while reducing 70.8% of the LLM generation latency. Notably, our Frame Compression Adjustment strategy is compatible with other token compression methods to further improve their performance. Our code is available at https://github.com/xuyang-liu16/VidCom2.

Method: GAIS uses two complementary modules: 1) Frame-level Gated Fusion (FGF) that adaptively integrates audio-visual features under textual guidance for fine-grained temporal frame selection, and 2) Semantic Variance-Scaled Perturbation (SVSP) that regularizes text embedding space with semantics-aware perturbation control.

Result: Extensive experiments on MSR-VTT, DiDeMo, LSMDC, and VATEX datasets show GAIS consistently outperforms strong baselines across multiple retrieval metrics while maintaining computational efficiency.

Conclusion: GAIS effectively strengthens multimodal alignment through representation-level fusion and regularization-level perturbation, demonstrating superior performance in text-to-video retrieval tasks.

Abstract: Text-to-video retrieval requires precise alignment between language and temporally rich audio-video signals. However, existing methods often emphasize visual cues while underutilizing audio semantics or relying on coarse fusion strategies, resulting in suboptimal multimodal representations. We introduce GAIS, a retrieval framework that strengthens multimodal alignment from both representation and regularization perspectives. First, a Frame-level Gated Fusion (FGF) module adaptively integrates audio-visual features under textual guidance, enabling fine-grained temporal selection of informative frames. Second, a Semantic Variance-Scaled Perturbation (SVSP) mechanism regularizes the text embedding space by controlling perturbation magnitude in a semantics-aware manner. These two modules are complementary: FGF minimizes modality gaps through selective fusion, while SVSP improves embedding stability and discrimination. Extensive experiments on MSR-VTT, DiDeMo, LSMDC, and VATEX demonstrate that GAIS consistently outperforms strong baselines across multiple retrieval metrics while maintaining notable computational efficiency.

Method: Initialize teacher and student from same pre-trained ViT, add random register tokens to student, use test-time augmentation on teacher to generate denoised embeddings, then optimize only a small subset of student weights.

Result: PH-Reg effectively reduces artifact tokens and improves segmentation and depth prediction performance under zero-shot and linear probing settings.

Conclusion: The proposed method enables efficient integration of register tokens into existing ViTs without full retraining, addressing artifact token issues while maintaining model performance.

Abstract: Vision Transformers (ViTs) have emerged as the dominant architecture for visual processing tasks, demonstrating excellent scalability with increased training data and model size. However, recent work has identified the emergence of artifact tokens in ViTs that are incongruous with local semantics. These anomalous tokens degrade ViT performance in tasks that require fine-grained localization or structural coherence. An effective mitigation of this issue is the addition of register tokens to ViTs, which implicitly “absorb” the artifact term during training. Given the availability of existing large-scale pre-trained ViTs, in this paper we seek add register tokens to existing models without needing to re-train from scratch, which is infeasible considering their size. Specifically, we propose Post Hoc Registers (PH-Reg), an efficient self-distillation method that integrates registers into an existing ViT without requiring additional labeled data and full retraining. PH-Reg initializes both teacher and student networks from the same pre-trained ViT. The teacher remains frozen and unmodified, while the student is augmented with randomly initialized register tokens. By applying test-time augmentation to the teacher’s inputs, we generate denoised dense embeddings free of artifacts, which are then used to optimize only a small subset of unlocked student weights. We show that our approach can effectively reduce the number of artifact tokens, improving the segmentation and depth prediction of the student ViT under zero-shot and linear probing.

Method: Proposed SlotMatch framework aligns teacher and student slots via cosine similarity without additional distillation objectives or auxiliary supervision. Uses simple knowledge distillation to transfer representations.

Result: Student model matches and outperforms teacher (SlotContrast) while using 3.6x fewer parameters and running up to 2.7x faster. Surpasses all other state-of-the-art unsupervised video segmentation models.

Conclusion: SlotMatch provides an effective and simple distillation framework that enables lightweight models to achieve superior performance in unsupervised video segmentation without complex additional losses.

Abstract: Unsupervised video segmentation is a challenging computer vision task, especially due to the lack of supervisory signals coupled with the complexity of visual scenes. To overcome this challenge, state-of-the-art models based on slot attention often have to rely on large and computationally expensive neural architectures. To this end, we propose a simple knowledge distillation framework that effectively transfers object-centric representations to a lightweight student. The proposed framework, called SlotMatch, aligns corresponding teacher and student slots via the cosine similarity, requiring no additional distillation objectives or auxiliary supervision. The simplicity of SlotMatch is confirmed via theoretical and empirical evidence, both indicating that integrating additional losses is redundant. We conduct experiments on three datasets to compare the state-of-the-art teacher model, SlotContrast, with our distilled student. The results show that our student based on SlotMatch matches and even outperforms its teacher, while using 3.6x less parameters and running up to 2.7x faster. Moreover, our student surpasses all other state-of-the-art unsupervised video segmentation models.

Method: Systematically explores ZO optimization from modality branch-wise to layer-wise levels, identifies vision modality’s higher variance, and proposes modality-aware ZO with gradient sign normalization and vision perturbation constraints.

Result: Extensive experiments on four benchmarks demonstrate state-of-the-art results, with ZO optimization enabling better escape from local minima during optimization.

Conclusion: ZO optimization effectively addresses local minima issues in PEFT-based VLCL, achieving superior performance through systematic exploration and modality-aware strategies.

Abstract: Vision-Language Continual Learning (VLCL) has attracted significant research attention for its robust capabilities, and the adoption of Parameter-Efficient Fine-Tuning (PEFT) strategies is enabling these models to achieve competitive performance with substantially reduced resource consumption. However, dominated First-Order (FO) optimization is prone to trap models in suboptimal local minima, especially in limited exploration subspace within PEFT. To overcome this challenge, this paper pioneers a systematic exploration of adopting Zeroth-Order (ZO) optimization for PEFT-based VLCL. We first identify the incompatibility of naive full-ZO adoption in VLCL due to optimization process instability. We then investigate the application of ZO optimization from a modality branch-wise to a fine-grained layer-wise across various training units to identify an optimal strategy. Besides, a key theoretical insight reveals that vision modality exhibit higher variance than language counterparts in VLCL during the ZO optimization process, and we propose a modality-aware ZO strategy, which adopts gradient sign normalization in ZO and constrains vision modality perturbation to further improve performance. Benefiting from the adoption of ZO optimization, PEFT-based VLCL fulfills better ability to escape local minima during the optimization process, extensive experiments on four benchmarks demonstrate that our method achieves state-of-the-art results.

Method: Proposes RelTopo with three-level relational modeling: 1) perception-level with relation-aware lane detector using geometry-biased self-attention and curve-guided cross-attention; 2) reasoning-level with geometry-enhanced L2L head and cross-view L2T head; 3) supervision-level with contrastive InfoNCE strategy for relational embeddings.

Result: Significant improvements on OpenLane-V2: +3.1 in DET$l$, +5.3 in TOP${ll}$, +4.9 in TOP$_{lt}$, and +4.4 overall in OLS, setting new state-of-the-art performance.

Conclusion: RelTopo demonstrates that systematic relational modeling across perception, reasoning, and supervision levels enables joint optimization of road detection and topology reasoning, significantly advancing autonomous driving capabilities.

Abstract: Accurate road topology reasoning is critical for autonomous driving, as it requires both perceiving road elements and understanding how lanes connect to each other (L2L) and to traffic elements (L2T). Existing methods often focus on either perception or L2L reasoning, leaving L2T underexplored and fall short of jointly optimizing perception and reasoning. Moreover, although topology prediction inherently involves relations, relational modeling itself is seldom incorporated into feature extraction or supervision. As humans naturally leverage contextual relationships to recognize road element and infer their connectivity, we posit that relational modeling can likewise benefit both perception and reasoning, and that these two tasks should be mutually enhancing. To this end, we propose RelTopo, a multi-level relational modeling approach that systematically integrates relational cues across three levels: 1) perception-level: a relation-aware lane detector with geometry-biased self-attention and curve-guided cross-attention enriches lane representations; 2) reasoning-level: relation-enhanced topology heads, including a geometry-enhanced L2L head and a cross-view L2T head, enhance topology inference via relational cues; and 3) supervision-level: a contrastive InfoNCE strategy regularizes relational embeddings. This design enables perception and reasoning to be optimized jointly. Extensive experiments on OpenLane-V2 demonstrate that RelTopo significantly improves both detection and topology reasoning, with gains of +3.1 in DET$l$, +5.3 in TOP${ll}$, +4.9 in TOP$_{lt}$, and +4.4 overall in OLS, setting a new state-of-the-art. Code will be released.

Method: Proposes 4D-VLA with sequential RGB-D inputs to align robot and scene coordinate systems, and introduces memory bank sampling for extracting informative frames from historical images.

Result: Significantly increases success rate over OpenVLA in both simulated and real-world experiments, and outperforms existing methods on the MV-Bench multi-view benchmark.

Conclusion: The integration of 4D information and memory bank sampling substantially enhances model performance, spatial perception, and generalization to novel views.

Abstract: Leveraging diverse robotic data for pretraining remains a critical challenge. Existing methods typically model the dataset’s action distribution using simple observations as inputs. However, these inputs are often incomplete, resulting in a dispersed conditional action distribution-an issue we refer to as coordinate system chaos and state chaos. This inconsistency significantly hampers pretraining efficiency. To address this, we propose 4D-VLA, a novel approach that effectively integrates 4D information into the input to mitigate these sources of chaos. Our model introduces depth and temporal information into visual features with sequential RGB-D inputs, aligning the coordinate systems of the robot and the scene. This alignment endows the model with strong spatiotemporal reasoning capabilities while minimizing training overhead. Additionally, we introduce memory bank sampling, a frame sampling strategy designed to extract informative frames from historical images, further improving effectiveness and efficiency. Experimental results demonstrate that our pretraining method and architectural components substantially enhance model performance. In both simulated and real-world experiments, our model achieves a significant increase in success rate over OpenVLA. To further assess spatial perception and generalization to novel views, we introduce MV-Bench, a multi-view simulation benchmark. Our model consistently outperforms existing methods, demonstrating stronger spatial understanding and adaptability.

Method: Proposes FIxLIP based on game theory using weighted Banzhaf interaction index for computational efficiency, extending evaluation metrics to second-order interactions.

Result: FIxLIP outperforms first-order attribution methods on MS COCO and ImageNet-1k benchmarks and enables effective comparison between different vision-language models.

Conclusion: Second-order interaction explanations provide superior insights into vision-language models’ behavior compared to traditional first-order methods, with FIxLIP offering computational advantages.

Abstract: Language-image pre-training (LIP) enables the development of vision-language models capable of zero-shot classification, localization, multimodal retrieval, and semantic understanding. Various explanation methods have been proposed to visualize the importance of input image-text pairs on the model’s similarity outputs. However, popular saliency maps are limited by capturing only first-order attributions, overlooking the complex cross-modal interactions intrinsic to such encoders. We introduce faithful interaction explanations of LIP models (FIxLIP) as a unified approach to decomposing the similarity in vision-language encoders. FIxLIP is rooted in game theory, where we analyze how using the weighted Banzhaf interaction index offers greater flexibility and improves computational efficiency over the Shapley interaction quantification framework. From a practical perspective, we propose how to naturally extend explanation evaluation metrics, such as the pointing game and area between the insertion/deletion curves, to second-order interaction explanations. Experiments on the MS COCO and ImageNet-1k benchmarks validate that second-order methods, such as FIxLIP, outperform first-order attribution methods. Beyond delivering high-quality explanations, we demonstrate the utility of FIxLIP in comparing different models, e.g. CLIP vs. SigLIP-2.

Method: Created dataset using hybrid annotation pipeline combining behavioral analysis, rule-based heuristics, and VLM semantic modeling with human verification. Built benchmark for evaluating personalized E2EAD models.

Result: Empirical evaluations show incorporating personalized driving preferences significantly improves behavioral alignment with human demonstrations in state-of-the-art architectures.

Conclusion: The introduced dataset and benchmark enable development and evaluation of personalized E2EAD models, demonstrating the importance of personalization for better human alignment.

Abstract: Personalization, while extensively studied in conventional autonomous driving pipelines, has been largely overlooked in the context of end-to-end autonomous driving (E2EAD), despite its critical role in fostering user trust, safety perception, and real-world adoption. A primary bottleneck is the absence of large-scale real-world datasets that systematically capture driving preferences, severely limiting the development and evaluation of personalized E2EAD models. In this work, we introduce the first large-scale real-world dataset explicitly curated for personalized E2EAD, integrating comprehensive scene topology with rich dynamic context derived from agent dynamics and semantics inferred via a fine-tuned vision-language model (VLM). We propose a hybrid annotation pipeline that combines behavioral analysis, rule-and-distribution-based heuristics, and subjective semantic modeling guided by VLM reasoning, with final refinement through human-in-the-loop verification. Building upon this dataset, we introduce the first standardized benchmark for systematically evaluating personalized E2EAD models. Empirical evaluations on state-of-the-art architectures demonstrate that incorporating personalized driving preferences significantly improves behavioral alignment with human demonstrations.

Method: Built UVLM benchmark through collaborative human-AI approach with careful data quality considerations: selected videos representing typical underwater challenges, ensured diversity in frame rates/resolutions/classes (419 marine animals), and designed 20 distinct task types categorized into biological/environmental observation with content/change-action subcategories.

Result: Fine-tuning VidLMs on UVLM significantly improves underwater world understanding while showing potential for slight improvements on existing in-air benchmarks like VideoMME and Perception text.

Conclusion: UVLM addresses the critical gap in underwater video understanding and demonstrates that specialized fine-tuning can enhance VidLM performance in challenging underwater environments while maintaining or slightly improving terrestrial capabilities.

Abstract: Recently, the remarkable success of large language models (LLMs) has achieved a profound impact on the field of artificial intelligence. Numerous advanced works based on LLMs have been proposed and applied in various scenarios. Among them, video language models (VidLMs) are particularly widely used. However, existing works primarily focus on terrestrial scenarios, overlooking the highly demanding application needs of underwater observation. To overcome this gap, we introduce UVLM, an under water observation benchmark which is build through a collaborative approach combining human expertise and AI models. To ensure data quality, we have conducted in-depth considerations from multiple perspectives. First, to address the unique challenges of underwater environments, we selected videos that represent typical underwater challenges including light variations, water turbidity, and diverse viewing angles to construct the dataset. Second, to ensure data diversity, the dataset covers a wide range of frame rates, resolutions, 419 classes of marine animals, and various static plants and terrains. Next, for task diversity, we adopted a structured design where observation targets are categorized into two major classes: biological and environmental. Each category includes content observation and change/action observation, totaling 20 distinct task types. Finally, we designed several challenging evaluation metrics to enable quantitative comparison and analysis of different methods. Experiments on two representative VidLMs demonstrate that fine-tuning VidLMs on UVLM significantly improves underwater world understanding while also showing potential for slight improvements on existing in-air VidLM benchmarks, such as VideoMME and Perception text. The dataset and prompt engineering will be released publicly.

Method: Proposes MOON with: (1) guided Mixture-of-Experts module for multimodal and aspect-specific content modeling; (2) core semantic region detection to mitigate background noise; (3) specialized negative sampling strategy for increased difficulty and diversity of negative samples.

Result: Demonstrates competitive zero-shot performance on both the proposed MBE benchmark and public datasets, showing strong generalization across cross-modal retrieval, product classification, and attribute prediction tasks. Case studies and visualizations confirm effectiveness.

Conclusion: MOON successfully addresses key challenges in product representation learning and establishes a new standard with the MBE benchmark, showcasing the potential of generative MLLMs for product understanding tasks.

Abstract: With the rapid advancement of e-commerce, exploring general representations rather than task-specific ones has attracted increasing research attention. For product understanding, although existing discriminative dual-flow architectures drive progress in this field, they inherently struggle to model the many-to-one alignment between multiple images and texts of products. Therefore, we argue that generative Multimodal Large Language Models (MLLMs) hold significant potential for improving product representation learning. Nevertheless, achieving this goal still remains non-trivial due to several key challenges: the lack of multimodal and aspect-aware modeling modules in typical LLMs; the common presence of background noise in product images; and the absence of a standard benchmark for evaluation. To address these issues, we propose the first generative MLLM-based model named MOON for product representation learning. Our method (1) employs a guided Mixture-of-Experts (MoE) module for targeted modeling of multimodal and aspect-specific product content; (2) effectively detects core semantic regions in product images to mitigate the distraction and interference caused by background noise; and (3) introduces the specialized negative sampling strategy to increase the difficulty and diversity of negative samples. In addition, we release a large-scale multimodal benchmark MBE for various product understanding tasks. Experimentally, our model demonstrates competitive zero-shot performance on both our benchmark and the public dataset, showcasing strong generalization across various downstream tasks, including cross-modal retrieval, product classification, and attribute prediction. Furthermore, the case study and visualization illustrate the effectiveness of MOON for product understanding.

Method: Deep unfolding of the LATINO Langevin MCMC sampler combined with model distillation to create few-step conditional models for posterior sampling.

Result: The proposed unfolded and distilled samplers achieve excellent accuracy and computational efficiency while maintaining flexibility to adapt to variations in forward models at inference time.

Conclusion: The framework successfully transforms diffusion model priors into efficient conditional samplers that outperform state-of-the-art methods in computational imaging tasks.

Abstract: Diffusion models (DMs) have emerged as powerful image priors in Bayesian computational imaging. Two primary strategies have been proposed for leveraging DMs in this context: Plug-and-Play methods, which are zero-shot and highly flexible but rely on approximations; and specialized conditional DMs, which achieve higher accuracy and faster inference for specific tasks through supervised training. In this work, we introduce a novel framework that integrates deep unfolding and model distillation to transform a DM image prior into a few-step conditional model for posterior sampling. A central innovation of our approach is the unfolding of a Markov chain Monte Carlo (MCMC) algorithm - specifically, the recently proposed LATINO Langevin sampler (Spagnoletti et al., 2025) - representing the first known instance of deep unfolding applied to a Monte Carlo sampling scheme. We demonstrate our proposed unfolded and distilled samplers through extensive experiments and comparisons with the state of the art, where they achieve excellent accuracy and computational efficiency, while retaining the flexibility to adapt to variations in the forward model at inference time.

Method: LENS uses a scalable reinforcement learning framework that jointly optimizes reasoning process and segmentation with unified rewards spanning sentence-, box-, and segment-level cues, encouraging informative CoT rationales while refining mask quality.

Result: Using Qwen2.5-VL-3B-Instruct model, LENS achieves average cIoU of 81.2% on RefCOCO, RefCOCO+, and RefCOCOg benchmarks, outperforming GLaMM by up to 5.6%.

Conclusion: RL-driven CoT reasoning significantly enhances text-prompted segmentation and offers a practical path toward more generalizable Segment Anything models.

Abstract: Text-prompted image segmentation enables fine-grained visual understanding and is critical for applications such as human-computer interaction and robotics. However, existing supervised fine-tuning methods typically ignore explicit chain-of-thought (CoT) reasoning at test time, which limits their ability to generalize to unseen prompts and domains. To address this issue, we introduce LENS, a scalable reinforcement-learning framework that jointly optimizes the reasoning process and segmentation in an end-to-end manner. We propose unified reinforcement-learning rewards that span sentence-, box-, and segment-level cues, encouraging the model to generate informative CoT rationales while refining mask quality. Using a publicly available 3-billion-parameter vision-language model, i.e., Qwen2.5-VL-3B-Instruct, LENS achieves an average cIoU of 81.2% on the RefCOCO, RefCOCO+, and RefCOCOg benchmarks, outperforming the strong fine-tuned method, i.e., GLaMM, by up to 5.6%. These results demonstrate that RL-driven CoT reasoning significantly enhances text-prompted segmentation and offers a practical path toward more generalizable Segment Anything models (SAM). Code is available at https://github.com/hustvl/LENS.

Method: Combines supervised fine-tuning, reinforcement learning, and Chain-of-Thought reasoning in an autoregressive multimodal framework that processes text and visual tokens uniformly. RL with a large multi-modal LLM verifier proved most effective.

Result: EARL performs competitively on diverse image editing tasks compared to strong baselines while using significantly less training data.

Conclusion: EARL advances the frontier of autoregressive multimodal models for image editing and demonstrates the effectiveness of RL-based approaches in this domain.

Abstract: While image generation techniques are now capable of producing high-quality images that respect prompts which span multiple sentences, the task of text-guided image editing remains a challenge. Even edit requests that consist of only a few words often fail to be executed correctly. We explore three strategies to enhance performance on a wide range of image editing tasks: supervised fine-tuning (SFT), reinforcement learning (RL), and Chain-of-Thought (CoT) reasoning. In order to study all these components in one consistent framework, we adopt an autoregressive multimodal model that processes textual and visual tokens in a unified manner. We find RL combined with a large multi-modal LLM verifier to be the most effective of these strategies. As a result, we release EARL: Editing with Autoregression and RL, a strong RL-based image editing model that performs competitively on a diverse range of edits compared to strong baselines, despite using much less training data. Thus, EARL pushes the frontier of autoregressive multimodal models on image editing. We release our code, training data, and trained models at https://github.com/mair-lab/EARL.

Method: Built upon a vision-language foundation model with region encoder and mask decoder for flexible region-level video interaction, plus an egocentric video pipeline for generating embodied cognition data.

Result: Achieves state-of-the-art performance in object property understanding, object segmentation, and spatial reasoning despite compact architecture.

Conclusion: RynnEC advances embodied agent development with its region-centric video paradigm and facilitates generalization across diverse embodied tasks, with code and benchmark publicly available.

Abstract: We introduce RynnEC, a video multimodal large language model designed for embodied cognition. Built upon a general-purpose vision-language foundation model, RynnEC incorporates a region encoder and a mask decoder, enabling flexible region-level video interaction. Despite its compact architecture, RynnEC achieves state-of-the-art performance in object property understanding, object segmentation, and spatial reasoning. Conceptually, it offers a region-centric video paradigm for the brain of embodied agents, providing fine-grained perception of the physical world and enabling more precise interactions. To mitigate the scarcity of annotated 3D datasets, we propose an egocentric video based pipeline for generating embodied cognition data. Furthermore, we introduce RynnEC-Bench, a region-centered benchmark for evaluating embodied cognitive capabilities. We anticipate that RynnEC will advance the development of general-purpose cognitive cores for embodied agents and facilitate generalization across diverse embodied tasks. The code, model checkpoints, and benchmark are available at: https://github.com/alibaba-damo-academy/RynnEC

Method: Developed Hierarchical Cascade Framework (HCF) with policy-driven quantization control and edge quantization principle for direct latent-space transformations across network nodes.

Result: Achieves up to 0.6dB PSNR gains, outperforms successive-compression by up to 5.56% BD-Rate on CLIC, saves up to 97.8% FLOPs, 96.5% GPU memory, and 90.0% execution time. Outperforms progressive methods by up to 12.64% BD-Rate on Kodak.

Conclusion: HCF provides superior rate-distortion performance and computational efficiency for distributed multi-stage image compression, enabling retraining-free cross-quality adaptation with significant BD-Rate reductions.

Abstract: Distributed multi-stage image compression – where visual content traverses multiple processing nodes under varying quality requirements – poses challenges. Progressive methods enable bitstream truncation but underutilize available compute resources; successive compression repeats costly pixel-domain operations and suffers cumulative quality loss and inefficiency; fixed-parameter models lack post-encoding flexibility. In this work, we developed the Hierarchical Cascade Framework (HCF) that achieves high rate-distortion performance and better computational efficiency through direct latent-space transformations across network nodes in distributed multi-stage image compression systems. Under HCF, we introduced policy-driven quantization control to optimize rate-distortion trade-offs, and established the edge quantization principle through differential entropy analysis. The configuration based on this principle demonstrates up to 0.6dB PSNR gains over other configurations. When comprehensively evaluated on the Kodak, CLIC, and CLIC2020-mobile datasets, HCF outperforms successive-compression methods by up to 5.56% BD-Rate in PSNR on CLIC, while saving up to 97.8% FLOPs, 96.5% GPU memory, and 90.0% execution time. It also outperforms state-of-the-art progressive compression methods by up to 12.64% BD-Rate on Kodak and enables retraining-free cross-quality adaptation with 7.13-10.87% BD-Rate reductions on CLIC2020-mobile.

Method: Proposes MonoDream framework with Unified Navigation Representation that aligns visual semantics and language-grounded action intent, and introduces Latent Panoramic Dreaming tasks to predict latent features of panoramic RGB-D observations from monocular input.

Result: Experiments on multiple VLN benchmarks show consistent improvement in monocular navigation performance and significant reduction of the gap with panoramic-based agents.

Conclusion: MonoDream enables more reliable monocular navigation by learning unified representations that capture panoramic spatial information through latent feature prediction, making VLN more practical for real-world deployment.

Abstract: Vision-Language Navigation (VLN) tasks often leverage panoramic RGB and depth inputs to provide rich spatial cues for action planning, but these sensors can be costly or less accessible in real-world deployments. Recent approaches based on Vision-Language Action (VLA) models achieve strong results with monocular input, yet they still lag behind methods using panoramic RGB-D information. We present MonoDream, a lightweight VLA framework that enables monocular agents to learn a Unified Navigation Representation (UNR). This shared feature representation jointly aligns navigation-relevant visual semantics (e.g., global layout, depth, and future cues) and language-grounded action intent, enabling more reliable action prediction. MonoDream further introduces Latent Panoramic Dreaming (LPD) tasks to supervise the UNR, which train the model to predict latent features of panoramic RGB and depth observations at both current and future steps based on only monocular input. Experiments on multiple VLN benchmarks show that MonoDream consistently improves monocular navigation performance and significantly narrows the gap with panoramic-based agents.

Method: Propose On-Need Declarative (OND) knowledge-based prompting that provides explicit image-label pair prompts to guide models in linking visual patterns with semantic concepts. Replace SAM’s prompt encoder with OND mechanism and fine-tune on historical maps.

Result: SMOL-MapSeg accurately segments user-defined classes and substantially outperforms baseline models. Demonstrates strong generalization even with minimal training data.

Conclusion: SMOL-MapSeg enables scalable and adaptable historical map analysis through flexible, concept-aware segmentation that supports arbitrary datasets and user-defined classes.

Abstract: Historical maps offer valuable insights into changes on Earth’s surface but pose challenges for modern segmentation models due to inconsistent visual styles and symbols. While deep learning models such as UNet and pre-trained foundation models perform well in domains like autonomous driving and medical imaging, they struggle with the variability of historical maps, where similar concepts appear in diverse forms. To address this issue, we propose On-Need Declarative (OND) knowledge-based prompting, a method that provides explicit image-label pair prompts to guide models in linking visual patterns with semantic concepts. This enables users to define and segment target concepts on demand, supporting flexible, concept-aware segmentation. Our approach replaces the prompt encoder of the Segment Anything Model (SAM) with the OND prompting mechanism and fine-tunes it on historical maps, creating SMOL-MapSeg (Show Me One Label). Unlike existing SAM-based fine-tuning methods that are class-agnostic or restricted to fixed classes, SMOL-MapSeg supports class-aware segmentation across arbitrary datasets. Experiments show that SMOL-MapSeg accurately segments user-defined classes and substantially outperforms baseline models. Furthermore, it demonstrates strong generalization even with minimal training data, highlighting its potential for scalable and adaptable historical map analysis.

Method: Created MobilTelesco dataset with sparse night-sky images from smartphone astrophotography and benchmarked several object detection models on it.

Result: Highlighted challenges faced by detection models under feature-deficient conditions in sparse night-sky images.

Conclusion: MobilTelesco dataset fills the gap for sparse signal detection and reveals limitations of current models in feature-deficient environments.

Abstract: Object detection models are typically trained on datasets like ImageNet, COCO, and PASCAL VOC, which focus on everyday objects. However, these lack signal sparsity found in non-commercial domains. MobilTelesco, a smartphone-based astrophotography dataset, addresses this by providing sparse night-sky images. We benchmark several detection models on it, highlighting challenges under feature-deficient conditions.

Method: The study analyzes Gaussian curvature as an invariant geometric quantity using the Middlebury stereo dataset, examining its correlation with performance of top stereo and monocular methods.

Result: Gaussian curvature provides a sparse and compact description of 3D surfaces. Strong correlation found between method performance rank and low total absolute Gaussian curvature.

Conclusion: Gaussian curvature can serve as a valuable geometric prior to enhance future 3D reconstruction algorithms, offering an explicit geometric model that current data-driven approaches lack.

Abstract: Recent advances in computer vision have predominantly relied on data-driven approaches that leverage deep learning and large-scale datasets. Deep neural networks have achieved remarkable success in tasks such as stereo matching and monocular depth reconstruction. However, these methods lack explicit models of 3D geometry that can be directly analyzed, transferred across modalities, or systematically modified for controlled experimentation. We investigate the role of Gaussian curvature in 3D surface modeling. Besides Gaussian curvature being an invariant quantity under change of observers or coordinate systems, we demonstrate using the Middlebury stereo dataset that it offers a sparse and compact description of 3D surfaces. Furthermore, we show a strong correlation between the performance rank of top state-of-the-art stereo and monocular methods and the low total absolute Gaussian curvature. We propose that this property can serve as a geometric prior to improve future 3D reconstruction algorithms.

Method: Model per-pixel depth as a mixture distribution to capture multiple plausible depths, shifting uncertainty from direct regression to mixture weights. Integrates into existing pipelines via variance-aware loss functions and uncertainty propagation.

Result: Achieves up to 35% higher boundary sharpness and improved point cloud quality compared to state-of-the-art baselines on KITTI and VKITTIv2 datasets.

Conclusion: The proposed mixture distribution approach enables crisp depth discontinuities using only self-supervision, significantly improving boundary sharpness and 3D point cloud quality.

Abstract: Accurate monocular depth estimation is crucial for 3D scene understanding, but existing methods often blur depth at object boundaries, introducing spurious intermediate 3D points. While achieving sharp edges usually requires very fine-grained supervision, our method produces crisp depth discontinuities using only self-supervision. Specifically, we model per-pixel depth as a mixture distribution, capturing multiple plausible depths and shifting uncertainty from direct regression to the mixture weights. This formulation integrates seamlessly into existing pipelines via variance-aware loss functions and uncertainty propagation. Extensive evaluations on KITTI and VKITTIv2 show that our method achieves up to 35% higher boundary sharpness and improves point cloud quality compared to state-of-the-art baselines.

Method: Combines prompt scaffolds, calibrated abstention, and standards-compliant PACS integration using four SLMs (Qwen2.5-VL, MiniCPM-V, Gemma 7B, LLaVA 7B) on NIH Chest X-ray dataset.

Result: Reflection-oriented prompts benefit lighter models while stronger baselines are less sensitive; framework operationalizes abstention, calibration error, oversight burden, and maps outputs to DICOM, HL7 v2, and FHIR standards.

Conclusion: Provides a prompt-first deployment framework with operations playbook for calibration, logging, change management, and clear pathway from pilot utilities to reader studies without over-claiming clinical validation.

Abstract: Small Language Models (SLMs) are a practical option for narrow, workflow-relevant medical imaging utilities where privacy, latency, and cost dominate. We present a governance-ready recipe that combines prompt scaffolds, calibrated abstention, and standards-compliant integration into Picture Archiving and Communication Systems (PACS). Our focus is the assistive task of AP/PA view tagging for chest radiographs. Using four deployable SLMs (Qwen2.5-VL, MiniCPM-V, Gemma 7B, LLaVA 7B) on NIH Chest X-ray, we provide illustrative evidence: reflection-oriented prompts benefit lighter models, whereas stronger baselines are less sensitive. Beyond accuracy, we operationalize abstention, expected calibration error, and oversight burden, and we map outputs to DICOM tags, HL7 v2 messages, and FHIR ImagingStudy. The contribution is a prompt-first deployment framework, an operations playbook for calibration, logging, and change management, and a clear pathway from pilot utilities to reader studies without over-claiming clinical validation. We additionally specify a human-factors RACI, stratified calibration for dataset shift, and an auditable evidence pack to support local governance reviews.

Method: Uses graph-learning with spatial graphs built from surface observables (elevation, velocity, mass balance), augmented with gradient features and polynomial trends. Employs Monte Carlo dropout for uncertainty modeling and hybrid loss combining confidence-weighted radar supervision with balanced regularization.

Result: Applied to three Greenland subregions, GraphTopoNet outperforms interpolation, convolutional, and graph-based baselines, reducing error by up to 60% while preserving fine-scale glacial features.

Conclusion: GraphTopoNet demonstrates how graph machine learning can convert sparse, uncertain geophysical observations into actionable knowledge at continental scale, improving reliability for operational climate modeling and policy support.

Abstract: Accurate maps of Greenland’s subglacial bed are essential for sea-level projections, but radar observations are sparse and uneven. We introduce GraphTopoNet, a graph-learning framework that fuses heterogeneous supervision and explicitly models uncertainty via Monte Carlo dropout. Spatial graphs built from surface observables (elevation, velocity, mass balance) are augmented with gradient features and polynomial trends to capture both local variability and broad structure. To handle data gaps, we employ a hybrid loss that combines confidence-weighted radar supervision with dynamically balanced regularization. Applied to three Greenland subregions, GraphTopoNet outperforms interpolation, convolutional, and graph-based baselines, reducing error by up to 60 percent while preserving fine-scale glacial features. The resulting bed maps improve reliability for operational modeling, supporting agencies engaged in climate forecasting and policy. More broadly, GraphTopoNet shows how graph machine learning can convert sparse, uncertain geophysical observations into actionable knowledge at continental scale.

Method: Decomposes inference into fine-grained sensing/encoding units for incremental computation, uses temporal aggregation module, adaptive multimodal configuration optimizer, and cross-modal speculative skipping mechanism.

Result: Significantly reduces end-to-end latency while maintaining high task accuracy across various system and data dynamics, validated on public datasets and UAV testbed.

Conclusion: MMEdge effectively enables efficient real-time multimodal inference on edge devices through pipelined design and adaptive optimization techniques.

Abstract: Real-time multimodal inference on resource-constrained edge devices is essential for applications such as autonomous driving, human-computer interaction, and mobile health. However, prior work often overlooks the tight coupling between sensing dynamics and model execution, as well as the complex inter-modality dependencies. In this paper, we propose MMEdge, an new on-device multi-modal inference framework based on pipelined sensing and encoding. Instead of waiting for complete sensor inputs, MMEdge decomposes the entire inference process into a sequence of fine-grained sensing and encoding units, allowing computation to proceed incrementally as data arrive. MMEdge also introduces a lightweight but effective temporal aggregation module that captures rich temporal dynamics across different pipelined units to maintain accuracy performance. Such pipelined design also opens up opportunities for fine-grained cross-modal optimization and early decision-making during inference. To further enhance system performance under resource variability and input data complexity, MMEdge incorporates an adaptive multimodal configuration optimizer that dynamically selects optimal sensing and model configurations for each modality under latency constraints, and a cross-modal speculative skipping mechanism that bypasses future units of slower modalities when early predictions reach sufficient confidence. We evaluate MMEdge using two public multimodal datasets and deploy it on a real-world unmanned aerial vehicle (UAV)-based multimodal testbed. The results show that MMEdge significantly reduces end-to-end latency while maintaining high task accuracy across various system and data dynamics.

Method: Transformer-based framework trained on automatically generated region correspondences, using edge-aware clustering and region matching to establish coherent correspondences during inference.

Result: Achieves 78.4-84.4% region-level accuracy across multiple datasets, demonstrating both high patch-level accuracy and strong region-level correspondence performance.

Conclusion: The method shows strong potential for real-world manga and animation applications by enabling robust region correspondence prediction in sparse line art images.

Abstract: Understanding region-wise correspondences between manga line art images is fundamental for high-level manga processing, supporting downstream tasks such as line art colorization and in-between frame generation. Unlike natural images that contain rich visual cues, manga line art consists only of sparse black-and-white strokes, making it challenging to determine which regions correspond across images. In this work, we introduce a new task: predicting region-wise correspondence between raw manga line art images without any annotations. To address this problem, we propose a Transformer-based framework trained on large-scale, automatically generated region correspondences. The model learns to suppress noisy matches and strengthen consistent structural relationships, resulting in robust patch-level feature alignment within and across images. During inference, our method segments each line art and establishes coherent region-level correspondences through edge-aware clustering and region matching. We construct manually annotated benchmarks for evaluation, and experiments across multiple datasets demonstrate both high patch-level accuracy and strong region-level correspondence performance, achieving 78.4-84.4% region-level accuracy. These results highlight the potential of our method for real-world manga and animation applications.

Method: Analyzes condition numbers for polynomial evaluation in different bases and bounds how various inner products provide norms for symbol variations.

Result: The study explores the trade-offs between basis selection and polynomial degree, examining computational efficiency through condition number analysis.

Conclusion: Different polynomial bases offer varying trade-offs between accuracy and computational cost for mathematical handwriting representation, with condition number analysis providing key insights for optimization.

Abstract: Previous work has made use of a parameterized plane curve polynomial representation for mathematical handwriting, with the polynomials represented in a Legendre or Legendre-Sobolev graded basis. This provides a compact geometric representation for the digital ink. Preliminary results have also been shown for Chebyshev and Chebyshev-Sobolev bases. This article explores the trade-offs between basis choice and polynomial degree to achieve accurate modeling with a low computational cost. To do this, we consider the condition number for polynomial evaluation in these bases and bound how the various inner products give norms for the variations between symbols.

Method: Leverages region-based segmentation to identify structurally significant regions, enabling selective downsampling of dense point clouds while preserving scene fidelity.

Result: Reduces Gaussian count by up to 50%, achieves comparable or superior PSNR and SSIM with marginal LPIPS degradation, enables faster training and lower memory footprint.

Conclusion: SDI-GS advances the practicality of 3DGS for constrained-view scenarios by balancing efficiency and quality through segmentation-driven initialization.

Abstract: Sparse-view synthesis remains a challenging problem due to the difficulty of recovering accurate geometry and appearance from limited observations. While recent advances in 3D Gaussian Splatting (3DGS) have enabled real-time rendering with competitive quality, existing pipelines often rely on Structure-from-Motion (SfM) for camera pose estimation, an approach that struggles in genuinely sparse-view settings. Moreover, several SfM-free methods replace SfM with multi-view stereo (MVS) models, but generate massive numbers of 3D Gaussians by back-projecting every pixel into 3D space, leading to high memory costs. We propose Segmentation-Driven Initialization for Gaussian Splatting (SDI-GS), a method that mitigates inefficiency by leveraging region-based segmentation to identify and retain only structurally significant regions. This enables selective downsampling of the dense point cloud, preserving scene fidelity while substantially reducing Gaussian count. Experiments across diverse benchmarks show that SDI-GS reduces Gaussian count by up to 50% and achieves comparable or superior rendering quality in PSNR and SSIM, with only marginal degradation in LPIPS. It further enables faster training and lower memory footprint, advancing the practicality of 3DGS for constrained-view scenarios.

Method: Following PRISMA-ScR guidelines, searched Google Scholar and dblp for English-language studies with quantitative results, with dual reviewer screening and charting across methods, datasets, and evaluation metrics.

Result: Reviewed 63 empirical studies (1990-2025) showing shift from static OCR-gates to vehicle-mounted cameras enabling precise monitoring, with end-to-end accuracy ranging 5-96%. Identified lack of public benchmark datasets and standardized terminology.

Conclusion: Proposes standardized terminology, advocates for open-access datasets/codebases, and suggests future directions including addressing vehicle-mounted camera challenges, synthetic data generation, unified end-to-end models, and contextless text recognition.

Abstract: Background: The standardisation of Intermodal Loading Units (ILUs), including containers, semi-trailers, and swap bodies, has transformed global trade, yet efficient and robust identification remains an operational bottleneck in ports and terminals. Objective: To map Computer Vision (CV) methods for ILU identification, clarify terminology, summarise the evolution of proposed approaches, and highlight research gaps, future directions and their potential effects on terminal operations. Methods: Following PRISMA-ScR, we searched Google Scholar and dblp for English-language studies with quantitative results. After dual reviewer screening, the studies were charted across methods, datasets, and evaluation metrics. Results: 63 empirical studies on CV-based solutions for the ILU identification task, published between 1990 and 2025 were reviewed. Methodological evolution of ILU identification solutions, datasets, evaluation of the proposed methods and future research directions are summarised. A shift from static (e.g. OCR-gates) to vehicle mounted camera setups, which enables precise monitoring is observed. The reported results for end-to-end accuracy range from 5% to 96%. Conclusions: We propose standardised terminology, advocate for open-access datasets, codebases and model weights to enable fair evaluation and define future work directions. The shift from static to dynamic camera settings introduces new challenges that have transformative potential for transportation and logistics. However, the lack of public benchmark datasets, open-access code, and standardised terminology hinders the advancements in this field. As for the future work, we suggest addressing the new challenges emerged from vehicle mounted cameras, exploring synthetic data generation, refining the multi-stage methods into unified end-to-end models to reduce complexity, and focusing on contextless text recognition.

Method: Developed models trained on the largest known dataset of post-disaster sUAS imagery (21,716 building damage labels) and operationally trained 91 disaster practitioners. Deployed best performing model during Hurricanes Debby and Helene.

Result: Successfully assessed 415 buildings in approximately 18 minutes during operational deployment, establishing the first state of practice for sUAS-based damage assessment systems.

Conclusion: This work contributes the first documented operational use of AI/ML for damage assessment during disasters and provides lessons learned for both AI/ML research and user communities.

Abstract: This paper presents the first AI/ML system for automating building damage assessment in uncrewed aerial systems (sUAS) imagery to be deployed operationally during federally declared disasters (Hurricanes Debby and Helene). In response to major disasters, sUAS teams are dispatched to collect imagery of the affected areas to assess damage; however, at recent disasters, teams collectively delivered between 47GB and 369GB of imagery per day, representing more imagery than can reasonably be transmitted or interpreted by subject matter experts in the disaster scene, thus delaying response efforts. To alleviate this data avalanche encountered in practice, computer vision and machine learning techniques are necessary. While prior work has been deployed to automatically assess damage in satellite imagery, there is no current state of practice for sUAS-based damage assessment systems, as all known work has been confined to academic settings. This work establishes the state of practice via the development and deployment of models for building damage assessment with sUAS imagery. The model development involved training on the largest known dataset of post-disaster sUAS aerial imagery, containing 21,716 building damage labels, and the operational training of 91 disaster practitioners. The best performing model was deployed during the responses to Hurricanes Debby and Helene, where it assessed a combined 415 buildings in approximately 18 minutes. This work contributes documentation of the actual use of AI/ML for damage assessment during a disaster and lessons learned to the benefit of the AI/ML research and user communities.

Method: Proposed Sa2VA-i that rectifies the identified inconsistencies in the original Sa2VA model while using the same checkpoints.

Result: Sa2VA-i sets new state-of-the-art results with improvements of +11.6 J&F on MeViS, +1.4 on Ref-YT-VOS, +3.3 on Ref-DAVIS and +4.1 on ReVOS. The Sa2VA-i-1B model performs on par with the original Sa2VA-26B model on MeViS.

Conclusion: This work demonstrates the importance of seemingly trivial implementation details and provides valuable insights for the referring video segmentation field.

Abstract: Sa2VA is a recent model for language-guided dense grounding in images and video that achieves state-of-the-art results on multiple segmentation benchmarks and that has become widely popular. However, we found that Sa2VA does not perform according to its full potential for referring video object segmentation tasks. We identify inconsistencies between training and inference procedures as the key factor holding it back. To mitigate this issue, we propose an improved version of Sa2VA, Sa2VA-i, that rectifies these issues and improves the results. In fact, Sa2VA-i sets a new state of the art for multiple video benchmarks and achieves improvements of up to +11.6 J&F on MeViS, +1.4 on Ref-YT-VOS, +3.3 on Ref-DAVIS and +4.1 on ReVOS using the same Sa2VA checkpoints. With our fixes, the Sa2VA-i-1B model even performs on par with the original Sa2VA-26B model on the MeViS benchmark. We hope that this work will show the importance of seemingly trivial implementation details and that it will provide valuable insights for the referring video segmentation field. We provide the code and updated models at https://github.com/kumuji/sa2va-i

Method: Combines rasterized Gaussian kernel rendering efficiency with edge-aware gating of Steered Mixture of Experts, replacing global iterative optimization with rasterized formulation.

Result: Achieves significantly faster parameter updates, more memory-efficient representations, and supports native super-resolution and denoising not possible with standard rasterized Gaussian approaches.

Conclusion: Provides new balance between computational efficiency and reconstruction fidelity for 2D image processing tasks.

Abstract: The Steered Mixture of Experts regression framework has demonstrated strong performance in image reconstruction, compression, denoising, and super-resolution. However, its high computational cost limits practical applications. This work introduces a rasterization-based optimization strategy that combines the efficiency of rasterized Gaussian kernel rendering with the edge-aware gating mechanism of the Steered Mixture of Experts. The proposed method is designed to accelerate two-dimensional image regression while maintaining the model’s inherent sparsity and reconstruction quality. By replacing global iterative optimization with a rasterized formulation, the method achieves significantly faster parameter updates and more memory-efficient model representations. In addition, the proposed framework supports applications such as native super-resolution and image denoising, which are not directly achievable with standard rasterized Gaussian kernel approaches. The combination of fast rasterized optimization with the edge-aware structure of the Steered Mixture of Experts provides a new balance between computational efficiency and reconstruction fidelity for two-dimensional image processing tasks.

Method: Built on ViT-GPT-2 backbone, uses multi-loss framework with: prompt-based cosine similarity loss for semantic alignment, CLIP-style loss for image-caption embedding alignment, and language-guided contrastive loss for task discriminability.

Result: Mitigates catastrophic forgetting while achieving better semantic caption alignment compared to state-of-the-art methods, with no additional inference overhead or prompts during generation.

Conclusion: The proposed multi-loss framework effectively addresses continual image captioning challenges through semantic guidance and contrastive alignment, achieving improved performance without inference-time costs.

Abstract: Generating accurate and coherent image captions in a continual learning setting remains a major challenge due to catastrophic forgetting and the difficulty of aligning evolving visual concepts with language over time. In this work, we propose a novel multi-loss framework for continual image captioning that integrates semantic guidance through prompt-based continual learning and contrastive alignment. Built upon a pretrained ViT-GPT-2 backbone, our approach combines standard cross-entropy loss with three additional components: (1) a prompt-based cosine similarity loss that aligns image embeddings with synthetically constructed prompts encoding objects, attributes, and actions; (2) a CLIP-style loss that promotes alignment between image embeddings and target caption embedding; and (3) a language-guided contrastive loss that employs a triplet loss to enhance class-level discriminability between tasks. Notably, our approach introduces no additional overhead at inference time and requires no prompts during caption generation. We find that this approach mitigates catastrophic forgetting, while achieving better semantic caption alignment compared to state-of-the-art methods. The code can be found via the following link: https://github.com/Gepardius/Taetz_Bordelius_Continual_ImageCaptioning.

Method: Uses diffusion transformer with LoRA-based training, position shift inference, partial parameter updates with reward feedback, and Mask-CFG for multi-character animation without training.

Result: Outperforms state-of-the-art approaches in quality, temporal coherence, and supports multi-character animation with high lip-sync accuracy.

Conclusion: Achieves efficient, cost-effective, high-quality audio-driven video generation for arbitrary lengths and multiple characters without specialized datasets.

Abstract: Recent advances in diffusion models have significantly improved audio-driven human video generation, surpassing traditional methods in both quality and controllability. However, existing approaches still face challenges in lip-sync accuracy, temporal coherence for long video generation, and multi-character animation. In this work, we propose a diffusion transformer (DiT)-based framework for generating lifelike talking videos of arbitrary length, and introduce a training-free method for multi-character audio-driven animation. First, we employ a LoRA-based training strategy combined with a position shift inference approach, which enables efficient long video generation while preserving the capabilities of the foundation model. Moreover, we combine partial parameter updates with reward feedback to enhance both lip synchronization and natural body motion. Finally, we propose a training-free approach, Mask Classifier-Free Guidance (Mask-CFG), for multi-character animation, which requires no specialized datasets or model modifications and supports audio-driven animation for three or more characters. Experimental results demonstrate that our method outperforms existing state-of-the-art approaches, achieving high-quality, temporally coherent, and multi-character audio-driven video generation in a simple, efficient, and cost-effective manner.

Method: Uses learnable HD estimation from image embeddings, HD-based loss with momentum-driven scheduling during training, and HD-guided rejection sampling during inference to select geometrically richer outputs.

Result: Achieves 39% improvement in output diversity compared to vanilla FGMs on ImageNet dataset while preserving comparable image quality.

Conclusion: First work to introduce Hausdorff Dimension into FGM, effectively enhancing diversity while providing theoretical contribution to FGM development.

Abstract: Improving the diversity of generated results while maintaining high visual quality remains a significant challenge in image generation tasks. Fractal Generative Models (FGMs) are efficient in generating high-quality images, but their inherent self-similarity limits the diversity of output images. To address this issue, we propose a novel approach based on the Hausdorff Dimension (HD), a widely recognized concept in fractal geometry used to quantify structural complexity, which aids in enhancing the diversity of generated outputs. To incorporate HD into FGM, we propose a learnable HD estimation method that predicts HD directly from image embeddings, addressing computational cost concerns. However, simply introducing HD into a hybrid loss is insufficient to enhance diversity in FGMs due to: 1) degradation of image quality, and 2) limited improvement in generation diversity. To this end, during training, we adopt an HD-based loss with a monotonic momentum-driven scheduling strategy to progressively optimize the hyperparameters, obtaining optimal diversity without sacrificing visual quality. Moreover, during inference, we employ HD-guided rejection sampling to select geometrically richer outputs. Extensive experiments on the ImageNet dataset demonstrate that our FGM-HD framework yields a 39% improvement in output diversity compared to vanilla FGMs, while preserving comparable image quality. To our knowledge, this is the very first work introducing HD into FGM. Our method effectively enhances the diversity of generated outputs while offering a principled theoretical contribution to FGM development.

Method: Evaluated 3D CNNs and LSTM networks on a dataset of 1,200 ASL signs across 50 classes, comparing accuracy, computational efficiency, and latency under similar training conditions. Also tested a hybrid 3D CNN-LSTM model.

Result: 3D CNNs achieved 92.4% recognition accuracy but required 3.2% more processing time per frame. LSTMs maintained 86.7% accuracy with significantly lower resource consumption. Hybrid model showed decent performance.

Conclusion: Context-dependent architecture selection is crucial for practical ASL recognition systems, with trade-offs between recognition precision and real-time operational requirements in edge computing environments.

Abstract: This study investigates the performance of 3D Convolutional Neural Networks (3D CNNs) and Long Short-Term Memory (LSTM) networks for real-time American Sign Language (ASL) recognition. Though 3D CNNs are good at spatiotemporal feature extraction from video sequences, LSTMs are optimized for modeling temporal dependencies in sequential data. We evaluate both architectures on a dataset containing 1,200 ASL signs across 50 classes, comparing their accuracy, computational efficiency, and latency under similar training conditions. Experimental results demonstrate that 3D CNNs achieve 92.4% recognition accuracy but require 3.2% more processing time per frame compared to LSTMs, which maintain 86.7% accuracy with significantly lower resource consumption. The hybrid 3D CNNLSTM model shows decent performance, which suggests that context-dependent architecture selection is crucial for practical implementation.This project provides professional benchmarks for developing assistive technologies, highlighting trade-offs between recognition precision and real-time operational requirements in edge computing environments.

Method: Knowledge-guided Contrastive Learning (KnowCoL) that combines images and text descriptions in shared semantic space using Wikidata’s structured information (descriptions, type hierarchies, relational context).

Result: Achieves 10.5% accuracy improvement on unseen entities compared to state-of-the-art while being 35 times smaller in model size, with significant gains for rare entities.

Conclusion: Combining visual, textual, and structured knowledge effectively improves open-domain visual entity recognition, especially for rare and unseen entities, demonstrating the value of knowledge grounding.

Abstract: Open-domain visual entity recognition aims to identify and link entities depicted in images to a vast and evolving set of real-world concepts, such as those found in Wikidata. Unlike conventional classification tasks with fixed label sets, it operates under open-set conditions, where most target entities are unseen during training and exhibit long-tail distributions. This makes the task inherently challenging due to limited supervision, high visual ambiguity, and the need for semantic disambiguation. We propose a Knowledge-guided Contrastive Learning (KnowCoL) framework that combines both images and text descriptions into a shared semantic space grounded by structured information from Wikidata. By abstracting visual and textual inputs to a conceptual level, the model leverages entity descriptions, type hierarchies, and relational context to support zero-shot entity recognition. We evaluate our approach on the OVEN benchmark, a large-scale open-domain visual recognition dataset with Wikidata IDs as the label space. Our experiments show that using visual, textual, and structured knowledge greatly improves accuracy, especially for rare and unseen entities. Our smallest model improves the accuracy on unseen entities by 10.5% compared to the state-of-the-art, despite being 35 times smaller.

Method: Train a discrete style codebook from images to extract embeddings, use these to condition a text-to-image diffusion model, then train an autoregressive style generator to synthesize novel style embeddings from numerical codes.

Result: CoTyle successfully generates diverse, consistent visual styles from numerical codes, demonstrating that a single code can effectively control style in image generation.

Conclusion: A style can be represented by one numerical code, enabling simple and diverse style generation without complex inputs, bridging a gap in open-source style generation research.

Abstract: Innovative visual stylization is a cornerstone of artistic creation, yet generating novel and consistent visual styles remains a significant challenge. Existing generative approaches typically rely on lengthy textual prompts, reference images, or parameter-efficient fine-tuning to guide style-aware image generation, but often struggle with style consistency, limited creativity, and complex style representations. In this paper, we affirm that a style is worth one numerical code by introducing the novel task, code-to-style image generation, which produces images with novel, consistent visual styles conditioned solely on a numerical style code. To date, this field has only been primarily explored by the industry (e.g., Midjourney), with no open-source research from the academic community. To fill this gap, we propose CoTyle, the first open-source method for this task. Specifically, we first train a discrete style codebook from a collection of images to extract style embeddings. These embeddings serve as conditions for a text-to-image diffusion model (T2I-DM) to generate stylistic images. Subsequently, we train an autoregressive style generator on the discrete style embeddings to model their distribution, allowing the synthesis of novel style embeddings. During inference, a numerical style code is mapped to a unique style embedding by the style generator, and this embedding guides the T2I-DM to generate images in the corresponding style. Unlike existing methods, our method offers unparalleled simplicity and diversity, unlocking a vast space of reproducible styles from minimal input. Extensive experiments validate that CoTyle effectively turns a numerical code into a style controller, demonstrating a style is worth one code.

Method: Introduces detailed human evaluation protocol for BEAT2 dataset, conducts large-scale crowdsourced evaluation of six gesture-generation models across motion realism and speech-gesture alignment dimensions.

Result: Newer models don’t consistently outperform earlier approaches; published claims of high quality may not hold under rigorous evaluation; field needs disentangled assessments of motion quality and multimodal alignment.

Conclusion: Standardized evaluation protocols are essential for accurate benchmarking and progress in gesture generation research; releases comprehensive evaluation dataset to drive future standardization.

Abstract: We review human evaluation practices in automated, speech-driven 3D gesture generation and find a lack of standardisation and frequent use of flawed experimental setups. This leads to a situation where it is impossible to know how different methods compare, or what the state of the art is. In order to address common shortcomings of evaluation design, and to standardise future user studies in gesture-generation works, we introduce a detailed human evaluation protocol for the widely-used BEAT2 motion-capture dataset. Using this protocol, we conduct large-scale crowdsourced evaluation to rank six recent gesture-generation models – each trained by its original authors – across two key evaluation dimensions: motion realism and speech-gesture alignment. Our results provide strong evidence that 1) newer models do not consistently outperform earlier approaches; 2) published claims of high motion realism or speech-gesture alignment may not hold up under rigorous evaluation; and 3) the field must adopt disentangled assessments of motion quality and multimodal alignment for accurate benchmarking in order to make progress. Finally, in order to drive standardisation and enable new evaluation research, we will release five hours of synthetic motion from the benchmarked models; over 750 rendered video stimuli from the user studies – enabling new evaluations without model reimplementation required – alongside our open-source rendering script, and the 16,000 pairwise human preference votes collected for our benchmark.

Method: Created PALM dataset with 13k hand scans from 263 subjects and 90k multi-view images capturing skin tone, age, and geometry variations. Developed PALM-Net using multi-subject prior learned via physically based inverse rendering.

Result: PALM provides large-scale diverse hand data. PALM-Net enables realistic, relightable single-image hand avatar personalization by learning geometry and material properties from the dataset.

Conclusion: PALM’s scale and diversity make it a valuable real-world resource for hand modeling and related research, addressing previous limitations in hand avatar creation.

Abstract: The ability to grasp objects, signal with gestures, and share emotion through touch all stem from the unique capabilities of human hands. Yet creating high-quality personalized hand avatars from images remains challenging due to complex geometry, appearance, and articulation, particularly under unconstrained lighting and limited views. Progress has also been limited by the lack of datasets that jointly provide accurate 3D geometry, high-resolution multiview imagery, and a diverse population of subjects. To address this, we present PALM, a large-scale dataset comprising 13k high-quality hand scans from 263 subjects and 90k multi-view images, capturing rich variation in skin tone, age, and geometry. To show its utility, we present a baseline PALM-Net, a multi-subject prior over hand geometry and material properties learned via physically based inverse rendering, enabling realistic, relightable single-image hand avatar personalization. PALM’s scale and diversity make it a valuable real-world resource for hand modeling and related research.

Method: Semi-supervised segmentation framework using sub-meter satellite images to detect individual refugee shelters, achieving 76.4% F1-score.

Result: WASH accessibility declined from 25 people per facility in 2022 to 29.4 in 2025, with women and girls experiencing reduced accessibility due to inadequate safety-related segregation.

Conclusion: Remote sensing and machine learning can detect inequality and inform equitable resource planning, highlighting the need for demand-responsive allocation strategies in humanitarian settings.

Abstract: Access to Water, Sanitation, and Hygiene (WASH) services remains a major public health concern in refugee camps. This study introduces a remote sensing-driven framework to quantify WASH accessibility-specifically to water pumps, latrines, and bathing cubicles-in the Rohingya camps of Cox’s Bazar, one of the world’s most densely populated displacement settings. Detecting refugee shelters in such emergent camps presents substantial challenges, primarily due to their dense spatial configuration and irregular geometric patterns. Using sub-meter satellite images, we develop a semi-supervised segmentation framework that achieves an F1-score of 76.4% in detecting individual refugee shelters. Applying the framework across multi-year data reveals declining WASH accessibility, driven by rapid refugee population growth and reduced facility availability, rising from 25 people per facility in 2022 to 29.4 in 2025. Gender-disaggregated analysis further shows that women and girls experience reduced accessibility, in scenarios with inadequate safety-related segregation in WASH facilities. These findings suggest the importance of demand-responsive allocation strategies that can identify areas with under-served populations-such as women and girls-and ensure that limited infrastructure serves the greatest number of people in settings with fixed or shrinking budgets. We also discuss the value of high-resolution remote sensing and machine learning to detect inequality and inform equitable resource planning in complex humanitarian environments.

Method: Proposes MMaDA-Parallel, a parallel multimodal diffusion framework that enables continuous bidirectional interaction between text and images throughout the denoising trajectory, trained with supervised finetuning and optimized by Parallel Reinforcement Learning (ParaRL) with semantic rewards along the trajectory.

Result: The model significantly improves cross-modal alignment and semantic consistency, achieving a 6.9% improvement in Output Alignment on ParaBench compared to state-of-the-art model Bagel.

Conclusion: MMaDA-Parallel establishes a more robust paradigm for thinking-aware image synthesis by addressing error propagation through parallel processing and trajectory-based reinforcement learning.

Abstract: While thinking-aware generation aims to improve performance on complex tasks, we identify a critical failure mode where existing sequential, autoregressive approaches can paradoxically degrade performance due to error propagation. To systematically analyze this issue, we propose ParaBench, a new benchmark designed to evaluate both text and image output modalities. Our analysis using ParaBench reveals that this performance degradation is strongly correlated with poor alignment between the generated reasoning and the final image. To resolve this, we propose a parallel multimodal diffusion framework, MMaDA-Parallel, that enables continuous, bidirectional interaction between text and images throughout the entire denoising trajectory. MMaDA-Parallel is trained with supervised finetuning and then further optimized by Parallel Reinforcement Learning (ParaRL), a novel strategy that applies semantic rewards along the trajectory to enforce cross-modal consistency. Experiments validate that our model significantly improves cross-modal alignment and semantic consistency, achieving a 6.9% improvement in Output Alignment on ParaBench compared to the state-of-the-art model, Bagel, establishing a more robust paradigm for thinking-aware image synthesis. Our code is open-sourced at https://github.com/tyfeld/MMaDA-Parallel

Method: Proposes a 3D VAE framework using UDFs with local-to-global architecture that partitions UDF into uniform subvolumes (UBlocks), combining 3D convolutions for local detail with sparse transformers for global coherence, plus Pad-Average strategy for smooth boundary transitions.

Result: Achieves state-of-the-art performance in reconstruction accuracy and generative quality, with superior surface smoothness and geometric flexibility, enabling seamless scaling to ultra-high resolutions up to 2048^3.

Conclusion: The UDF-based 3D VAE with local-to-global architecture provides a robust solution for generating high-fidelity 3D content with complex topologies, overcoming fundamental limitations of previous methods.

Abstract: Generating high-fidelity 3D contents remains a fundamental challenge due to the complexity of representing arbitrary topologies-such as open surfaces and intricate internal structures-while preserving geometric details. Prevailing methods based on signed distance fields (SDFs) are hampered by costly watertight preprocessing and struggle with non-manifold geometries, while point-cloud representations often suffer from sampling artifacts and surface discontinuities. To overcome these limitations, we propose a novel 3D variational autoencoder (VAE) framework built upon unsigned distance fields (UDFs)-a more robust and computationally efficient representation that naturally handles complex and incomplete shapes. Our core innovation is a local-to-global (LoG) architecture that processes the UDF by partitioning it into uniform subvolumes, termed UBlocks. This architecture couples 3D convolutions for capturing local detail with sparse transformers for enforcing global coherence. A Pad-Average strategy further ensures smooth transitions at subvolume boundaries during reconstruction. This modular design enables seamless scaling to ultra-high resolutions up to $2048^3$-a regime previously unattainable for 3D VAEs. Experiments demonstrate state-of-the-art performance in both reconstruction accuracy and generative quality, yielding superior surface smoothness and geometric flexibility.

Method: Divides topology prediction into three phases: Hierarchical Prior Extractor (HPE) for spatial and sequential priors, Region-Focused Decoder (RFD) for fine-grained query construction, and Robust Boundary-Point Topology Reasoning (RBTR) with denoising strategy.

Result: Achieves state-of-the-art performance on OpenLane-V2 benchmark with OLS scores of 48.0 on subsetA and 45.4 on subsetB.

Conclusion: By integrating spatial and sequential priors into fine-grained queries and applying topological denoising, TopoFG precisely models complex lane structures and delivers trustworthy topology predictions for autonomous driving.

Abstract: Precise modeling of lane topology is essential for autonomous driving, as it directly impacts navigation and control decisions. Existing methods typically represent each lane with a single query and infer topological connectivity based on the similarity between lane queries. However, this kind of design struggles to accurately model complex lane structures, leading to unreliable topology prediction. In this view, we propose a Fine-Grained lane topology reasoning framework (TopoFG). It divides the procedure from bird’s-eye-view (BEV) features to topology prediction via fine-grained queries into three phases, i.e., Hierarchical Prior Extractor (HPE), Region-Focused Decoder (RFD), and Robust Boundary-Point Topology Reasoning (RBTR). Specifically, HPE extracts global spatial priors from the BEV mask and local sequential priors from in-lane keypoint sequences to guide subsequent fine-grained query modeling. RFD constructs fine-grained queries by integrating the spatial and sequential priors. It then samples reference points in RoI regions of the mask and applies cross-attention with BEV features to refine the query representations of each lane. RBTR models lane connectivity based on boundary-point query features and further employs a topological denoising strategy to reduce matching ambiguity. By integrating spatial and sequential priors into fine-grained queries and applying a denoising strategy to boundary-point topology reasoning, our method precisely models complex lane structures and delivers trustworthy topology predictions. Extensive experiments on the OpenLane-V2 benchmark demonstrate that TopoFG achieves new state-of-the-art performance, with an OLS of 48.0 on subsetA and 45.4 on subsetB.

Method: Couples unrolled TV solver with data-driven Lambda Mapping Network (LambdaNet) that predicts per-pixel regularization maps, trained end-to-end for joint optimization of reconstruction and regularization.

Result: Experiments on DeepLesion dataset show consistent gains: +2.9 dB PSNR and +6% SSIM on average over classical TV and FBP+U-Net methods.

Conclusion: LTV provides interpretable alternative to black-box CNNs and serves as basis for 3D and data-consistency-driven reconstruction.

Abstract: Although Total Variation (TV) performs well in noise reduction and edge preservation on images, its dependence on the lambda parameter limits its efficiency and makes it difficult to use effectively. In this study, we present a Learnable Total Variation (LTV) framework that couples an unrolled TV solver with a data-driven Lambda Mapping Network (LambdaNet) predicting a per-pixel regularization map. The pipeline is trained end-to-end so that reconstruction and regularization are optimized jointly, yielding spatially adaptive smoothing: strong in homogeneous regions, relaxed near anatomical boundaries. Experiments on the DeepLesion dataset, using a realistic noise model adapted from the LoDoPaB-CT methodology, show consistent gains over classical TV and FBP+U-Net: +2.9 dB PSNR and +6% SSIM on average. LTV provides an interpretable alternative to black-box CNNs and a basis for 3D and data-consistency-driven reconstruction.

Method: Created using GPT-4o-assisted vision-language pipeline with human-in-the-loop verification, annotating images from Argoverse 1, Cityscapes, KITTI, and nuScenes with 28 categories covering environment, road geometry, vehicle behavior, and severity scales.

Result: Provides attribute co-occurrence graphs, JSONL records, and reproducible baselines including LoRA-tuned Qwen2-VL-2B model with deterministic decoding, evaluated via accuracy, F1 scores, and severity MAE/RMSE metrics.

Conclusion: CAR-Scenes enables explainable, data-centric workflows for intelligent vehicles by providing comprehensive annotations, analysis tools, and baseline models for vision-language understanding in autonomous driving scenarios.

Abstract: CAR-Scenes is a frame-level dataset for autonomous driving that enables training and evaluation of vision-language models (VLMs) for interpretable, scene-level understanding. We annotate 5,192 images drawn from Argoverse 1, Cityscapes, KITTI, and nuScenes using a 28-key category/sub-category knowledge base covering environment, road geometry, background-vehicle behavior, ego-vehicle behavior, vulnerable road users, sensor states, and a discrete severity scale (1-10), totaling 350+ leaf attributes. Labels are produced by a GPT-4o-assisted vision-language pipeline with human-in-the-loop verification; we release the exact prompts, post-processing rules, and per-field baseline model performance. CAR-Scenes also provides attribute co-occurrence graphs and JSONL records that support semantic retrieval, dataset triage, and risk-aware scenario mining across sources. To calibrate task difficulty, we include reproducible, non-benchmark baselines, notably a LoRA-tuned Qwen2-VL-2B with deterministic decoding, evaluated via scalar accuracy, micro-averaged F1 for list attributes, and severity MAE/RMSE on a fixed validation split. We publicly release the annotation and analysis scripts, including graph construction and evaluation scripts, to enable explainable, data-centric workflows for future intelligent vehicles. Dataset: https://github.com/Croquembouche/CAR-Scenes

Method: Two-stage approach: (1) Fine-tune pose-aware diffusion model using ControlNet for pose estimation and CPPL loss; (2) Distill 3D avatar using NeRF representation with canonical SMPL-X space sampling and Multi-Resolution 3D-SDS.

Result: Achieves 48x speed-up (5 minutes personalization), outperforms SOTA in reconstruction fidelity, detail preservation, and robustness to occlusions/truncations. Preserves high-frequency textures and handles occlusions correctly.

Conclusion: PFAvatar advances practical 3D avatar generation from real-world OOTD albums and supports downstream applications like virtual try-on, animation, and human video reenactment.

Abstract: We propose PFAvatar (Pose-Fusion Avatar), a new method that reconstructs high-quality 3D avatars from Outfit of the Day(OOTD) photos, which exhibit diverse poses, occlusions, and complex backgrounds. Our method consists of two stages: (1) fine-tuning a pose-aware diffusion model from few-shot OOTD examples and (2) distilling a 3D avatar represented by a neural radiance field (NeRF). In the first stage, unlike previous methods that segment images into assets (e.g., garments, accessories) for 3D assembly, which is prone to inconsistency, we avoid decomposition and directly model the full-body appearance. By integrating a pre-trained ControlNet for pose estimation and a novel Condition Prior Preservation Loss (CPPL), our method enables end-to-end learning of fine details while mitigating language drift in few-shot training. Our method completes personalization in just 5 minutes, achieving a 48x speed-up compared to previous approaches. In the second stage, we introduce a NeRF-based avatar representation optimized by canonical SMPL-X space sampling and Multi-Resolution 3D-SDS. Compared to mesh-based representations that suffer from resolution-dependent discretization and erroneous occluded geometry, our continuous radiance field can preserve high-frequency textures (e.g., hair) and handle occlusions correctly through transmittance. Experiments demonstrate that PFAvatar outperforms state-of-the-art methods in terms of reconstruction fidelity, detail preservation, and robustness to occlusions/truncations, advancing practical 3D avatar generation from real-world OOTD albums. In addition, the reconstructed 3D avatar supports downstream applications such as virtual try-on, animation, and human video reenactment, further demonstrating the versatility and practical value of our approach.

Method: Hybrid State-Space Vision-Language Model using Liquid State-Space Dynamics instead of attention, with Token-Grid Correlation Module for visual grounding via FiLM conditioning.

Result: Achieves accurate fine-grained understanding with significantly improved efficiency across multiple benchmarks while maintaining linear-time inference.

Conclusion: Viper-F1 demonstrates that efficient state-space models can replace attention mechanisms in multimodal systems while preserving fine-grained visual reasoning capabilities.

Abstract: Recent advances in multimodal large language models (MLLMs) have enabled impressive progress in vision-language understanding, yet their high computational cost limits deployment in resource-constrained scenarios such as robotic manipulation, personal assistants, and smart cameras. Most existing methods rely on Transformer-based cross-attention, whose quadratic complexity hinders efficiency. Moreover, small vision-language models often struggle to precisely capture fine-grained, task-relevant visual regions, leading to degraded performance on fine-grained reasoning tasks that limit their effectiveness in the real world. To address these issues, we introduce Viper-F1, a Hybrid State-Space Vision-Language Model that replaces attention with efficient Liquid State-Space Dynamics. To further enhance visual grounding, we propose a Token-Grid Correlation Module, which computes lightweight correlations between text tokens and image patches and modulates the state-space dynamics via FiLM conditioning. This enables the model to selectively emphasize visual regions relevant to the textual prompt while maintaining linear-time inference. Experimental results across multiple benchmarks demonstrate that Viper-F1 achieves accurate, fine-grained understanding with significantly improved efficiency.

Method: Proposes UNSEEN framework that integrates with existing pruning methods, uses generalization-based scoring, scales to multi-step scenarios with incremental selection, and optimizes coreset quality dynamically using models trained on varying coresets.

Result: Significantly outperforms SOTA methods on CIFAR-10, CIFAR-100, and ImageNet-1K. On ImageNet-1K, achieves lossless performance while reducing training data by 30%.

Conclusion: UNSEEN effectively addresses limitations of fitting-centric dataset pruning by adopting a generalization perspective, enabling more discriminative sample selection and achieving superior performance with substantial data reduction.

Abstract: The growing scale of datasets in deep learning has introduced significant computational challenges. Dataset pruning addresses this challenge by constructing a compact but informative coreset from the full dataset with comparable performance. Previous approaches typically establish scoring metrics based on specific criteria to identify representative samples. However, these methods predominantly rely on sample scores obtained from the model’s performance during the training (i.e., fitting) phase. As scoring models achieve near-optimal performance on training data, such fitting-centric approaches induce a dense distribution of sample scores within a narrow numerical range. This concentration reduces the distinction between samples and hinders effective selection. To address this challenge, we conduct dataset pruning from the perspective of generalization, i.e., scoring samples based on models not exposed to them during training. We propose a plug-and-play framework, UNSEEN, which can be integrated into existing dataset pruning methods. Additionally, conventional score-based methods are single-step and rely on models trained solely on the complete dataset, providing limited perspective on the importance of samples. To address this limitation, we scale UNSEEN to multi-step scenarios and propose an incremental selection technique through scoring models trained on varying coresets, and optimize the quality of the coreset dynamically. Extensive experiments demonstrate that our method significantly outperforms existing state-of-the-art (SOTA) methods on CIFAR-10, CIFAR-100, and ImageNet-1K. Notably, on ImageNet-1K, UNSEEN achieves lossless performance while reducing training data by 30%.

Method: STONE uses a parameterizable spherical trigger with configurable spatial properties to create a dynamic key space, enabling a single trigger to control multiple output labels. The approach is theoretically grounded through Neural Tangent Kernel (NTK) analysis.

Result: Extensive evaluations show high attack success rates (up to 100%) with no loss in clean-data accuracy, demonstrating the effectiveness of the one-to-N backdoor attack framework.

Conclusion: This work establishes the first foundational benchmark for multi-target threats in 3D vision, providing crucial insights for securing future intelligent systems against sophisticated backdoor attacks.

Abstract: Backdoor attacks pose a critical threat to deep learning, especially in safety-sensitive 3D domains such as autonomous driving and robotics. Despite their potency, existing attacks on 3D point clouds are limited to a static one-to-one paradigm, leaving the more flexible one-to-N backdoor threat largely unexplored and without a theoretical or practical foundation. We address this by introducing STONE (Spherical Trigger One-to-N Backdoor Enabling), the first framework that instantiates this threat through a configurable spherical trigger. Its parameterizable spatial properties create a dynamic key space, enabling a single trigger to control multiple output labels. Theoretically, we ground STONE through Neural Tangent Kernel (NTK) analysis, providing the first formal basis for one-to-N mappings in 3D models. Empirically, extensive evaluations show high attack success rate (up to 100%) with no loss in clean-data accuracy. This work establishes a foundational benchmark for multi-target threats in 3D vision, crucial for securing future intelligent systems.

Method: GEODE augments main VLM with two plug-and-play modules: Decoupled Rationale Module (DRM) for spatial co-processing and cross-attention alignment, and Direct Regression Head (DRH) using “Embedding-as-Value” paradigm for precise continuous regression.

Result: The 1.5B parameter model achieves state-of-the-art spatial reasoning performance rivaling 7B+ models, functioning as a high-level semantic dispatcher.

Conclusion: GEODE successfully resolves the dual-bottleneck in 3D spatial reasoning for VLMs through its decoupled architecture, enabling efficient and precise 3D understanding with smaller model size.

Abstract: Existing Vision Language Models (VLMs) architecturally rooted in “flatland” perception, fundamentally struggle to comprehend real-world 3D spatial intelligence. This failure stems from a dual-bottleneck: input-stage conflict between computationally exorbitant geometric-aware encoders and superficial 2D-only features, and output-stage misalignment where discrete tokenizers are structurally incapable of producing precise, continuous numerical values. To break this impasse, we introduce GEODE (Geometric-Output and Decoupled-Input Engine), a novel architecture that resolves this dual-bottleneck by decoupling 3D reasoning from numerical generation. GEODE augments main VLM with two specialized, plug-and-play modules: Decoupled Rationale Module (DRM) that acts as spatial co-processor, aligning explicit 3D data with 2D visual features via cross-attention and distilling spatial Chain-of-Thought (CoT) logic into injectable Rationale Tokens; and Direct Regression Head (DRH), an “Embedding-as-Value” paradigm which routes specialized control tokens to a lightweight MLP for precise, continuous regression of scalars and 3D bounding boxes. The synergy of these modules allows our 1.5B parameter model to function as a high-level semantic dispatcher, achieving state-of-the-art spatial reasoning performance that rivals 7B+ models.

Method: Proposes Reference-Frame × Granularity (RFxG) taxonomy framework and four novel faithfulness metrics to systematically evaluate explanation quality across both dimensions, applied to ten saliency methods, four model architectures, and three datasets.

Result: Demonstrates critical limitations in existing evaluation metrics that prioritize pointwise faithfulness while neglecting contrastive reasoning and semantic granularity, providing comprehensive evaluation framework.

Conclusion: Advocates shift toward user-intent-driven evaluation, providing conceptual foundation and practical tools to develop visual explanations that are faithful to model behavior and aligned with human understanding complexity.

Abstract: Saliency maps are widely used for visual explanations in deep learning, but a fundamental lack of consensus persists regarding their intended purpose and alignment with diverse user queries. This ambiguity hinders the effective evaluation and practical utility of explanation methods. We address this gap by introducing the Reference-Frame $\times$ Granularity (RFxG) taxonomy, a principled conceptual framework that organizes saliency explanations along two essential axes:Reference-Frame: Distinguishing between pointwise (“Why this prediction?”) and contrastive (“Why this and not an alternative?”) explanations. Granularity: Ranging from fine-grained class-level (e.g., “Why Husky?”) to coarse-grained group-level (e.g., “Why Dog?”) interpretations. Using the RFxG lens, we demonstrate critical limitations in existing evaluation metrics, which overwhelmingly prioritize pointwise faithfulness while neglecting contrastive reasoning and semantic granularity. To systematically assess explanation quality across both RFxG dimensions, we propose four novel faithfulness metrics. Our comprehensive evaluation framework applies these metrics to ten state-of-the-art saliency methods, four model architectures, and three datasets. By advocating a shift toward user-intent-driven evaluation, our work provides both the conceptual foundation and the practical tools necessary to develop visual explanations that are not only faithful to the underlying model behavior but are also meaningfully aligned with the complexity of human understanding and inquiry.

Method: Incorporates Hierarchical Multimodal Compressor for joint encoding of visual, textual, and layout information into fixed-length sequences, plus Streaming Abstention mechanism with entropy-based uncertainty calibration for scalable processing.

Result: Matches or surpasses state-of-the-art methods while using 82% fewer visual tokens, 75% fewer parameters, and 71% lower latency across multiple long multimodal document benchmarks.

Conclusion: DocSLM delivers reliable multimodal document understanding on lightweight edge devices with significantly reduced resource requirements.

Abstract: Large Vision-Language Models (LVLMs) have demonstrated strong multimodal reasoning capabilities on long and complex documents. However, their high memory footprint makes them impractical for deployment on resource-constrained edge devices. We present DocSLM, an efficient Small Vision-Language Model designed for long-document understanding under constrained memory resources. DocSLM incorporates a Hierarchical Multimodal Compressor that jointly encodes visual, textual, and layout information from each page into a fixed-length sequence, greatly reducing memory consumption while preserving both local and global semantics. To enable scalable processing over arbitrarily long inputs, we introduce a Streaming Abstention mechanism that operates on document segments sequentially and filters low-confidence responses using an entropy-based uncertainty calibrator. Across multiple long multimodal document benchmarks, DocSLM matches or surpasses state-of-the-art methods while using 82% fewer visual tokens, 75% fewer parameters, and 71% lower latency, delivering reliable multimodal document understanding on lightweight edge devices. Code is available in the supplementary material.

Method: Extends DDCM by replacing random noise in backward process with noise sampled from specific sets according to predefined rules, making it compatible with multiple diffusion model variants.

Result: Successfully generalized DDCM to various diffusion models and achieved improved performance on CIFAR-10 and LSUN Bedroom datasets.

Conclusion: gDDCM effectively extends image compression capabilities to mainstream diffusion models beyond just DDPM, demonstrating broader applicability and enhanced performance.

Abstract: Recently, the Denoising Diffusion Codebook Models (DDCM) was proposed. DDCM leverages the Denoising Diffusion Probabilistic Model (DDPM) and replaces the random noise in the backward process with noise sampled from specific sets according to a predefined rule, thereby enabling image compression. However, DDCM cannot be applied to methods other than DDPM. In this paper, we propose the generalized Denoising Diffusion Compression Model (gDDCM), which extends DDCM to mainstream diffusion models and their variants, including DDPM, Score-Based Models, Consistency Models, and Rectified Flow. We evaluate our method on CIFAR-10 and LSUN Bedroom datasets. Experimental results demonstrate that our approach successfully generalizes DDCM to the aforementioned models and achieves improved performance.

Method: Uses a multistep process: Hough Line Transform for edge detection, projective transform for board alignment, segmentation into 64 squares, and piece classification using a residual CNN with 13 classes (6 white pieces, 6 black pieces, empty square).

Result: Trained and evaluated on Chess Recognition Dataset (ChessReD) with 10,800 annotated smartphone images captured under diverse conditions, achieving accurate piece recognition and FEN conversion.

Conclusion: The system successfully converts physical chessboard images to FEN notation, enabling integration with online chess engines to provide optimal move suggestions for physical chess games.

Abstract: Chess has experienced a large increase in viewership since the pandemic, driven largely by the accessibility of online learning platforms. However, no equivalent assistance exists for physical chess games, creating a divide between analog and digital chess experiences. This paper presents CVChess, a deep learning framework for converting chessboard images to Forsyth-Edwards Notation (FEN), which is later input into online chess engines to provide you with the best next move. Our approach employs a convolutional neural network (CNN) with residual layers to perform piece recognition from smartphone camera images. The system processes RGB images of a physical chess board through a multistep process: image preprocessing using the Hough Line Transform for edge detection, projective transform to achieve a top-down board alignment, segmentation into 64 individual squares, and piece classification into 13 classes (6 unique white pieces, 6 unique black pieces and an empty square) using the residual CNN. Residual connections help retain low-level visual features while enabling deeper feature extraction, improving accuracy and stability during training. We train and evaluate our model using the Chess Recognition Dataset (ChessReD), containing 10,800 annotated smartphone images captured under diverse lighting conditions and angles. The resulting classifications are encoded as an FEN string, which can be fed into a chess engine to generate the most optimal move

Method: Proposes Foresee pipeline with type-prior-driven strategy and Flexible Feature Detector (FFD) module to handle copy-move manipulations, enabling training-free operation and lightweight inference while unleashing vanilla MLLMs’ potential in forensic domain.

Result: Extensive experiments show Foresee achieves superior localization accuracy and richer textual explanations than existing MLLM-based methods, with stronger generalization across various tampering types including copy-move, splicing, removal, local enhancement, deepfake, and AIGC-based editing.

Conclusion: Foresee demonstrates that vanilla MLLMs have inherent potential for image forgery analysis without additional training, providing an effective training-free solution that outperforms existing methods in both accuracy and explanatory richness.

Abstract: With the rapid advancement of artificial intelligence-generated content (AIGC) technologies, including multimodal large language models (MLLMs) and diffusion models, image generation and manipulation have become remarkably effortless. Existing image forgery detection and localization (IFDL) methods often struggle to generalize across diverse datasets and offer limited interpretability. Nowadays, MLLMs demonstrate strong generalization potential across diverse vision-language tasks, and some studies introduce this capability to IFDL via large-scale training. However, such approaches cost considerable computational resources, while failing to reveal the inherent generalization potential of vanilla MLLMs to address this problem. Inspired by this observation, we propose Foresee, a training-free MLLM-based pipeline tailored for image forgery analysis. It eliminates the need for additional training and enables a lightweight inference process, while surpassing existing MLLM-based methods in both tamper localization accuracy and the richness of textual explanations. Foresee employs a type-prior-driven strategy and utilizes a Flexible Feature Detector (FFD) module to specifically handle copy-move manipulations, thereby effectively unleashing the potential of vanilla MLLMs in the forensic domain. Extensive experiments demonstrate that our approach simultaneously achieves superior localization accuracy and provides more comprehensive textual explanations. Moreover, Foresee exhibits stronger generalization capability, outperforming existing IFDL methods across various tampering types, including copy-move, splicing, removal, local enhancement, deepfake, and AIGC-based editing. The code will be released in the final version.

Method: Geometry-guided Multi-View Diffusion Model with multi-view geometry extraction (depth, normal maps, segmentation), decoupled geometry-enhanced attention, adaptive learning strategy, iterative refinement, and dynamic geometry intensity adjustment.

Result: The model generates images that are consistent across views while preserving rich details, improving overall image quality and detail restoration.

Conclusion: The proposed geometry-guided approach effectively addresses cross-view consistency and detail preservation challenges in multi-view image generation, producing realistic and coherent results.

Abstract: Multi-view image generation holds significant application value in computer vision, particularly in domains like 3D reconstruction, virtual reality, and augmented reality. Most existing methods, which rely on extending single images, face notable computational challenges in maintaining cross-view consistency and generating high-resolution outputs. To address these issues, we propose the Geometry-guided Multi-View Diffusion Model, which incorporates mechanisms for extracting multi-view geometric information and adjusting the intensity of geometric features to generate images that are both consistent across views and rich in detail. Specifically, we design a multi-view geometry information extraction module that leverages depth maps, normal maps, and foreground segmentation masks to construct a shared geometric structure, ensuring shape and structural consistency across different views. To enhance consistency and detail restoration during generation, we develop a decoupled geometry-enhanced attention mechanism that strengthens feature focus on key geometric details, thereby improving overall image quality and detail preservation. Furthermore, we apply an adaptive learning strategy that fine-tunes the model to better capture spatial relationships and visual coherence between the generated views, ensuring realistic results. Our model also incorporates an iterative refinement process that progressively improves the output quality through multiple stages of image generation. Finally, a dynamic geometry information intensity adjustment mechanism is proposed to adaptively regulate the influence of geometric data, optimizing overall quality while ensuring the naturalness of generated images. More details can be found on the project page: https://sobeymil.github.io/GeoMVD.com.

Method: Teacher-student knowledge distillation using frozen DINOv3 teacher trained on clean images to provide stable representations, distilled to student trained on blurred images for consistent performance under motion degradation.

Result: Achieves state-of-the-art performance under both motion-blurred and clean conditions, demonstrating improved generalization and real-world applicability.

Conclusion: The proposed framework effectively addresses motion blur challenges in AI-generated image detection through knowledge distillation, enhancing practical deployment in real-world scenarios.

Abstract: With growing concerns over image authenticity and digital safety, the field of AI-generated image (AIGI) detection has progressed rapidly. Yet, most AIGI detectors still struggle under real-world degradations, particularly motion blur, which frequently occurs in handheld photography, fast motion, and compressed video. Such blur distorts fine textures and suppresses high-frequency artifacts, causing severe performance drops in real-world settings. We address this limitation with a blur-robust AIGI detection framework based on teacher-student knowledge distillation. A high-capacity teacher (DINOv3), trained on clean (i.e., sharp) images, provides stable and semantically rich representations that serve as a reference for learning. By freezing the teacher to maintain its generalization ability, we distill its feature and logit responses from sharp images to a student trained on blurred counterparts, enabling the student to produce consistent representations under motion degradation. Extensive experiments benchmarks show that our method achieves state-of-the-art performance under both motion-blurred and clean conditions, demonstrating improved generalization and real-world applicability. Source codes will be released at: https://github.com/JiaLiangShen/Dino-Detect-for-blur-robust-AIGC-Detection.

Method: Proposes a universal framework with vision-language fusion, global context incorporation, task-aware text embedding, dual-branch diffusion for concurrent head-hand movement prediction, target indicators for specific joint waypoints, and hand-object interaction state prediction.

Result: Achieves state-of-the-art performance on multiple datasets in multi-dimensional and multi-target hand motion forecasting, and demonstrates impressive human-robot policy transfer for robotic manipulation and effective feature enhancement for action tasks.

Conclusion: Uni-Hand successfully addresses key limitations in hand motion forecasting and establishes the first downstream task evaluation benchmarks, showing strong real-world applicability for human-robot interaction and augmented reality applications.

Abstract: Forecasting how human hands move in egocentric views is critical for applications like augmented reality and human-robot policy transfer. Recently, several hand trajectory prediction (HTP) methods have been developed to generate future possible hand waypoints, which still suffer from insufficient prediction targets, inherent modality gaps, entangled hand-head motion, and limited validation in downstream tasks. To address these limitations, we present a universal hand motion forecasting framework considering multi-modal input, multi-dimensional and multi-target prediction patterns, and multi-task affordances for downstream applications. We harmonize multiple modalities by vision-language fusion, global context incorporation, and task-aware text embedding injection, to forecast hand waypoints in both 2D and 3D spaces. A novel dual-branch diffusion is proposed to concurrently predict human head and hand movements, capturing their motion synergy in egocentric vision. By introducing target indicators, the prediction model can forecast the specific joint waypoints of the wrist or the fingers, besides the widely studied hand center points. In addition, we enable Uni-Hand to additionally predict hand-object interaction states (contact/separation) to facilitate downstream tasks better. As the first work to incorporate downstream task evaluation in the literature, we build novel benchmarks to assess the real-world applicability of hand motion forecasting algorithms. The experimental results on multiple publicly available datasets and our newly proposed benchmarks demonstrate that Uni-Hand achieves the state-of-the-art performance in multi-dimensional and multi-target hand motion forecasting. Extensive validation in multiple downstream tasks also presents its impressive human-robot policy transfer to enable robotic manipulation, and effective feature enhancement for action anticipation/recognition.

Method: Uses Arti4URDF pipeline with 3D point clouds, LLM prior knowledge, and URDF-oriented prompts to identify articulable objects and reconstruct executable URDF models while preserving geometry.

Result: Achieves state-of-the-art performance across 3D simulated objects, full scenes, and real-world scans, preserving object geometry and correctly capturing interactivity.

Conclusion: Provides a practical path for building interactive, robot-ready simulation environments directly from existing 3D assets.

Abstract: Building interactive simulators and scalable robot-learning environments requires a large number of articulated assets. However, most existing 3D assets in simulation are rigid, and manually converting them into articulated objects is extremely labor- and cost-intensive. This raises a natural question: can we automatically identify articulable objects in a scene and convert them into articulated assets directly? In this paper, we present ArtiWorld, a scene-aware pipeline that localizes candidate articulable objects from textual scene descriptions and reconstructs executable URDF models that preserve the original geometry. At the core of this pipeline is Arti4URDF, which leverages 3D point cloud, prior knowledge of a large language model (LLM), and a URDF-oriented prompt design to rapidly convert rigid objects into interactive URDF-based articulated objects while maintaining their 3D shape. We evaluate ArtiWorld at three levels: 3D simulated objects, full 3D simulated scenes, and real-world scan scenes. Across all three settings, our method consistently outperforms existing approaches and achieves state-of-the-art performance, while preserving object geometry and correctly capturing object interactivity to produce usable URDF-based articulated models. This provides a practical path toward building interactive, robot-ready simulation environments directly from existing 3D assets. Code and data will be released.

Method: Proposes GeoUniPS with Light-Geometry Dual-Branch Encoder that extracts multi-illumination cues and geometric priors from frozen 3D reconstruction models, and introduces PS-Perp dataset with realistic perspective projection.

Result: GeoUniPS achieves state-of-the-art performance across multiple datasets, both quantitatively and qualitatively, especially in complex in-the-wild scenes.

Conclusion: Integrating geometric priors from 3D foundation models and addressing perspective projection limitations significantly improves Universal Photometric Stereo performance in challenging real-world conditions.

Abstract: Universal Photometric Stereo is a promising approach for recovering surface normals without strict lighting assumptions. However, it struggles when multi-illumination cues are unreliable, such as under biased lighting or in shadows or self-occluded regions of complex in-the-wild scenes. We propose GeoUniPS, a universal photometric stereo network that integrates synthetic supervision with high-level geometric priors from large-scale 3D reconstruction models pretrained on massive in-the-wild data. Our key insight is that these 3D reconstruction models serve as visual-geometry foundation models, inherently encoding rich geometric knowledge of real scenes. To leverage this, we design a Light-Geometry Dual-Branch Encoder that extracts both multi-illumination cues and geometric priors from the frozen 3D reconstruction model. We also address the limitations of the conventional orthographic projection assumption by introducing the PS-Perp dataset with realistic perspective projection to enable learning of spatially varying view directions. Extensive experiments demonstrate that GeoUniPS delivers state-of-the-arts performance across multiple datasets, both quantitatively and qualitatively, especially in the complex in-the-wild scenes.

Method: Uses MobileSAM for instance segmentation, 3D Semantic Group Attention for object-centric feature aggregation, Global Similarity-Guided Attention for handling segmentation errors, and Instance-aware Local Diffusion for feature refinement in BEV space.

Result: Achieves state-of-the-art performance with mIoU scores of 17.40 on SemanticKITTI and 20.28 on SSCBench-KITTI-360 benchmarks.

Conclusion: Object-centric decomposition enables more accurate semantic occupancy prediction compared to traditional ego-centric approaches.

Abstract: Vision-based 3D Semantic Scene Completion (SSC) has received growing attention due to its potential in autonomous driving. While most existing approaches follow an ego-centric paradigm by aggregating and diffusing features over the entire scene, they often overlook fine-grained object-level details, leading to semantic and geometric ambiguities, especially in complex environments. To address this limitation, we propose Ocean, an object-centric prediction framework that decomposes the scene into individual object instances to enable more accurate semantic occupancy prediction. Specifically, we first employ a lightweight segmentation model, MobileSAM, to extract instance masks from the input image. Then, we introduce a 3D Semantic Group Attention module that leverages linear attention to aggregate object-centric features in 3D space. To handle segmentation errors and missing instances, we further design a Global Similarity-Guided Attention module that leverages segmentation features for global interaction. Finally, we propose an Instance-aware Local Diffusion module that improves instance features through a generative process and subsequently refines the scene representation in the BEV space. Extensive experiments on the SemanticKITTI and SSCBench-KITTI360 benchmarks demonstrate that Ocean achieves state-of-the-art performance, with mIoU scores of 17.40 and 20.28, respectively.

Method: Proposes a dual-branch structure that explicitly decouples scene perception (without ego status in BEV encoder) and ego-driven reasoning, with adaptive fusion via a scene-aware fusion module. Includes path attention mechanism, BEV unidirectional distillation, and autoregressive online mapping as auxiliary tasks.

Result: Achieves state-of-the-art open-loop planning performance on nuScenes dataset, significantly mitigates ego status over-reliance, and demonstrates impressive generalization across diverse scenarios.

Conclusion: AdaptiveAD provides an effective architectural solution to ego status over-reliance in autonomous driving systems, enabling more robust and generalizable planning through explicit decoupling and adaptive fusion strategies.

Abstract: Modular design of planning-oriented autonomous driving has markedly advanced end-to-end systems. However, existing architectures remain constrained by an over-reliance on ego status, hindering generalization and robust scene understanding. We identify the root cause as an inherent design within these architectures that allows ego status to be easily leveraged as a shortcut. Specifically, the premature fusion of ego status in the upstream BEV encoder allows an information flow from this strong prior to dominate the downstream planning module. To address this challenge, we propose AdaptiveAD, an architectural-level solution based on a multi-context fusion strategy. Its core is a dual-branch structure that explicitly decouples scene perception and ego status. One branch performs scene-driven reasoning based on multi-task learning, but with ego status deliberately omitted from the BEV encoder, while the other conducts ego-driven reasoning based solely on the planning task. A scene-aware fusion module then adaptively integrates the complementary decisions from the two branches to form the final planning trajectory. To ensure this decoupling does not compromise multi-task learning, we introduce a path attention mechanism for ego-BEV interaction and add two targeted auxiliary tasks: BEV unidirectional distillation and autoregressive online mapping. Extensive evaluations on the nuScenes dataset demonstrate that AdaptiveAD achieves state-of-the-art open-loop planning performance. Crucially, it significantly mitigates the over-reliance on ego status and exhibits impressive generalization capabilities across diverse scenarios.

Method: Developed MedGEN-Bench with 6,422 expert-validated image-text pairs spanning 6 imaging modalities, 16 clinical tasks, and 28 subtasks, structured into Visual Question Answering, Image Editing, and Contextual Multimodal Generation formats with contextually intertwined instructions requiring cross-modal reasoning.

Result: Systematically evaluated 10 compositional frameworks, 3 unified models, and 5 VLMs using a three-tier assessment framework integrating pixel-level metrics, semantic text analysis, and expert-guided clinical relevance scoring.

Conclusion: MedGEN-Bench provides a comprehensive benchmark that advances medical AI research by enabling evaluation of multimodal systems capable of sophisticated cross-modal reasoning and generative outputs, moving beyond multiple-choice formats to better reflect clinical needs.

Abstract: As Vision-Language Models (VLMs) increasingly gain traction in medical applications, clinicians are progressively expecting AI systems not only to generate textual diagnoses but also to produce corresponding medical images that integrate seamlessly into authentic clinical workflows. Despite the growing interest, existing medical visual benchmarks present notable limitations. They often rely on ambiguous queries that lack sufficient relevance to image content, oversimplify complex diagnostic reasoning into closed-ended shortcuts, and adopt a text-centric evaluation paradigm that overlooks the importance of image generation capabilities. To address these challenges, we introduce MedGEN-Bench, a comprehensive multimodal benchmark designed to advance medical AI research. MedGEN-Bench comprises 6,422 expert-validated image-text pairs spanning six imaging modalities, 16 clinical tasks, and 28 subtasks. It is structured into three distinct formats: Visual Question Answering, Image Editing, and Contextual Multimodal Generation. What sets MedGEN-Bench apart is its focus on contextually intertwined instructions that necessitate sophisticated cross-modal reasoning and open-ended generative outputs, moving beyond the constraints of multiple-choice formats. To evaluate the performance of existing systems, we employ a novel three-tier assessment framework that integrates pixel-level metrics, semantic text analysis, and expert-guided clinical relevance scoring. Using this framework, we systematically assess 10 compositional frameworks, 3 unified models, and 5 VLMs.

Method: Mixture-of-Experts framework with adaptive routing mechanism that dynamically selects among multiple YOLOv9-T expert models for different features.

Result: Achieves higher mean Average Precision (mAP) and Average Recall (AR) compared to a single YOLOv9-T model.

Conclusion: The Mixture-of-Experts approach with adaptive routing successfully improves object detection performance by leveraging specialized feature processing from multiple experts.

Abstract: This paper presents a novel Mixture-of-Experts framework for object detection, incorporating adaptive routing among multiple YOLOv9-T experts to enable dynamic feature specialization and achieve higher mean Average Precision (mAP) and Average Recall (AR) compared to a single YOLOv9-T model.

Method: Developed a saliency-aware attack framework called Chromatic Perturbation Module that systematically crafts adversarial examples by altering color contrast between foreground and background to disrupt explanation fidelity in federated learning settings.

Result: The attack reduces peak activation overlap in Grad-CAM explanations by up to 35% while preserving classification accuracy above 96% across multiple datasets. Standard training pipelines fail to detect or mitigate this explanation degradation.

Conclusion: Interpretability itself can be an attack surface in machine learning systems, particularly in federated learning where subtle color perturbations accumulate stealthily and persistently poison global model feature attributions.

Abstract: As machine learning models are increasingly deployed in safety-critical domains, visual explanation techniques have become essential tools for supporting transparency. In this work, we reveal a new class of attacks that compromise model interpretability without affecting accuracy. Specifically, we show that small color perturbations applied by adversarial clients in a federated learning setting can shift a model’s saliency maps away from semantically meaningful regions while keeping the prediction unchanged. The proposed saliency-aware attack framework, called Chromatic Perturbation Module, systematically crafts adversarial examples by altering the color contrast between foreground and background in a way that disrupts explanation fidelity. These perturbations accumulate across training rounds, poisoning the global model’s internal feature attributions in a stealthy and persistent manner. Our findings challenge a common assumption in model auditing that correct predictions imply faithful explanations and demonstrate that interpretability itself can be an attack surface. We evaluate this vulnerability across multiple datasets and show that standard training pipelines are insufficient to detect or mitigate explanation degradation, especially in the federated learning setting, where subtle color perturbations are harder to discern. Our attack reduces peak activation overlap in Grad-CAM explanations by up to 35% while preserving classification accuracy above 96% on all evaluated datasets.

Method: Introduced SpatiaLite, a fully synthetic benchmark to measure spatial reasoning accuracy and efficiency. Proposed Imagery Driven Framework (IDF) for data synthesis and training to build internal world models.

Result: Three key findings: 1) VLMs rely heavily on linguistic representations, failing at visual-centric spatial tasks; 2) VLMs are inefficient with token usage growing rapidly with complexity; 3) IDF can implicitly construct internal world models for better spatial reasoning.

Conclusion: This work identifies spatial reasoning limitations in advanced VLMs, reveals their current patterns and inefficiencies, and provides a framework (IDF) to guide future improvements in spatial reasoning capabilities.

Abstract: Large language models (LLMs) and vision language models (VLMs), such as DeepSeek R1,OpenAI o3, and Gemini 2.5 Pro, have demonstrated remarkable reasoning capabilities across logical inference, problem solving, and decision making. However, spatial reasoning:a fundamental component of human cognition that includes mental rotation, navigation, and spatial relationship comprehension remains a significant challenge for current advanced VLMs. We hypothesize that imagination, the internal simulation of spatial states, is the dominant reasoning mechanism within a spatial world model. To test this hypothesis and systematically probe current VLM spatial reasoning mechanisms, we introduce SpatiaLite, a fully synthetic benchmark that jointly measures spatial reasoning accuracy and reasoning efficiency. Comprehensive experiments reveal three key findings. First, advanced VLMs predominantly rely on linguistic representations for reasoning and imagination, resulting in significant deficiencies on visual centric tasks that demand perceptual spatial relations and 3D geometry transformations such as mental rotation or projection prediction. Second, advanced VLMs exhibit severe inefficiency in their current spatial reasoning mechanisms, with token usage growing rapidly as transformation complexity increases. Third, we propose an Imagery Driven Framework (IDF) for data synthesis and training, which can implicitly construct an internal world model that is critical for spatial reasoning in VLMs. Building on SpatiaLite, this work delineates the spatial reasoning limits and patterns of advanced VLMs, identifies key shortcomings, and informs future advances

Method: Proposes KANGURA framework that formulates prediction as function decomposition using KAN-based representation learning, geometry-disentangled representation learning to separate structural variations, and unified attention mechanisms to enhance critical geometric regions.

Result: Outperforms 15+ SOTA models on ModelNet40 with 92.7% accuracy and achieves 97% accuracy in real-world MFC anode structure optimization.

Conclusion: KANGURA establishes a robust framework for 3D geometric modeling, enabling optimization of complex structures in advanced manufacturing and quality-driven engineering applications.

Abstract: Microbial Fuel Cells (MFCs) offer a promising pathway for sustainable energy generation by converting organic matter into electricity through microbial processes. A key factor influencing MFC performance is the anode structure, where design and material properties play a crucial role. Existing predictive models struggle to capture the complex geometric dependencies necessary to optimize these structures. To solve this problem, we propose KANGURA: Kolmogorov-Arnold Network-Based Geometry-Aware Learning with Unified Representation Attention. KANGURA introduces a new approach to three-dimensional (3D) machine learning modeling. It formulates prediction as a function decomposition problem, where Kolmogorov-Arnold Network (KAN)- based representation learning reconstructs geometric relationships without a conventional multi- layer perceptron (MLP). To refine spatial understanding, geometry-disentangled representation learning separates structural variations into interpretable components, while unified attention mechanisms dynamically enhance critical geometric regions. Experimental results demonstrate that KANGURA outperforms over 15 state-of-the-art (SOTA) models on the ModelNet40 benchmark dataset, achieving 92.7% accuracy, and excels in a real-world MFC anode structure problem with 97% accuracy. This establishes KANGURA as a robust framework for 3D geometric modeling, unlocking new possibilities for optimizing complex structures in advanced manufacturing and quality-driven engineering applications.

Method: Uses Router-Planner-Coordinator workflow with optional Evaluator, prompt-engineered agents coordinated via LangGraph’s StateGraph, hybrid memory (FAISS + SQLite), and incremental indexing with file hashing.

Result: Successfully applied to multiple research areas at Argonne National Laboratory including waste-to-products research, energy process safety, and general scientific assistance, demonstrating cross-domain applicability.

Conclusion: AISAC provides a flexible, transparent multi-agent framework for scientific workflows that can be customized for various research domains while maintaining provenance tracking and adaptability.

Abstract: AI Scientific Assistant Core (AISAC) is an integrated multi-agent system developed at Argonne National Laboratory for scientific and engineering workflows. AISAC builds on established technologies - LangGraph for orchestration, FAISS for vector search, and SQLite for persistence - and integrates them into a unified system prototype focused on transparency, provenance tracking, and scientific adaptability. The system implements a Router-Planner-Coordinator workflow and an optional Evaluator role, using prompt-engineered agents coordinated via LangGraph’s StateGraph and supported by helper agents such as a Researcher. Each role is defined through custom system prompts that enforce structured JSON outputs. A hybrid memory approach (FAISS + SQLite) enables both semantic retrieval and structured conversation history. An incremental indexing strategy based on file hashing minimizes redundant re-embedding when scientific corpora evolve. A configuration-driven project bootstrap layer allows research teams to customize tools, prompts, and data sources without modifying core code. All agent decisions, tool invocations, and retrievals are logged and visualized through a custom Gradio interface, providing step-by-step transparency for each reasoning episode. The authors have applied AISAC to multiple research areas at Argonne, including specialized deployments for waste-to-products research and energy process safety, as well as general-purpose scientific assistance, demonstrating its cross-domain applicability.

Method: Used simple random-gene null benchmarks to test three hypotheses: DDR capacity predicting p53 response, OGT/CDO1 predicting radiation-response modules, and a 12-gene signature predicting prostate cancer recurrence after radiotherapy.

Result: Hypothesis 1 (DDR-p53) was not supported (r = -0.40, p = 0.76), Hypothesis 2 showed weak OGT association (r = 0.23, p = 0.34) but strong CDO1 association (r = 0.70, p = 0.0039), Hypothesis 3 achieved C-index 0.61 (p = 0.017) with non-unique effect size.

Conclusion: AI scientists can generate useful scientific ideas but require rigorous auditing against appropriate null models, as they produce a mix of well-supported discoveries, uncertain results, and false hypotheses.

Abstract: Agentic AI “scientists” now use language models to search the literature, run analyses, and generate hypotheses. We evaluate KOSMOS, an autonomous AI scientist, on three problems in radiation biology using simple random-gene null benchmarks. Hypothesis 1: baseline DNA damage response (DDR) capacity across cell lines predicts the p53 transcriptional response after irradiation (GSE30240). Hypothesis 2: baseline expression of OGT and CDO1 predicts the strength of repressed and induced radiation-response modules in breast cancer cells (GSE59732). Hypothesis 3: a 12-gene expression signature predicts biochemical recurrence-free survival after prostate radiotherapy plus androgen deprivation therapy (GSE116918). The DDR-p53 hypothesis was not supported: DDR score and p53 response were weakly negatively correlated (Spearman rho = -0.40, p = 0.76), indistinguishable from random five-gene scores. OGT showed only a weak association (r = 0.23, p = 0.34), whereas CDO1 was a clear outlier (r = 0.70, empirical p = 0.0039). The 12-gene signature achieved a concordance index of 0.61 (p = 0.017) but a non-unique effect size. Overall, KOSMOS produced one well-supported discovery, one plausible but uncertain result, and one false hypothesis, illustrating that AI scientists can generate useful ideas but require rigorous auditing against appropriate null models.

Method: Combines mean-field dynamics from network science with randomized utility models from economics to create a generative model of LLM interactions with latent truth states.

Result: Established conditions for fixed point existence/uniqueness in LLM networks and experimentally analyzed 100 open-source LLMs across various network conditions and datasets.

Conclusion: Network interactions significantly impact LLM reliability, with hallucinations propagating through the system and affecting overall collaborative question-answering performance.

Abstract: In this paper, we model and analyze how a network of interacting LLMs performs collaborative question-answering (CQA) in order to estimate a ground truth given a distributed set of documents. This problem is interesting because LLMs often hallucinate when direct evidence to answer a question is lacking, and these effects become more pronounced in a network of interacting LLMs. The hallucination spreads, causing previously accurate LLMs to hallucinate. We study interacting LLMs and their hallucination by combining novel ideas of mean-field dynamics (MFD) from network science and the randomized utility model from economics to construct a useful generative model. We model the LLM with a latent state that indicates if it is truthful or not with respect to the ground truth, and extend a tractable analytical model considering an MFD to model the diffusion of information in a directed network of LLMs. To specify the probabilities that govern the dynamics of the MFD, we propose a randomized utility model. For a network of LLMs, where each LLM has two possible latent states, we posit sufficient conditions for the existence and uniqueness of a fixed point and analyze the behavior of the fixed point in terms of the incentive (e.g., test-time compute) given to individual LLMs. We experimentally study and analyze the behavior of a network of $100$ open-source LLMs with respect to data heterogeneity, node capability, network structure, and sensitivity to framing on multiple semi-synthetic datasets.

Method: Investigates extension challenges using minimal examples, proposes alternative solution strategies, and evaluates them theoretically and through computational studies.

Result: Identifies challenges in extending the methodology to semi-Markovian models and develops alternative strategies to address these challenges.

Conclusion: The paper successfully extends the counterfactual probability bounding approach to semi-Markovian SCMs, enabling representation of confounding relationships through alternative solution strategies.

Abstract: Recently, Bjøru et al. proposed a novel divide-and-conquer algorithm for bounding counterfactual probabilities in structural causal models (SCMs). They assumed that the SCMs were learned from purely observational data, leading to an imprecise characterization of the marginal distributions of exogenous variables. Their method leveraged the canonical representation of structural equations to decompose a general SCM with high-cardinality exogenous variables into a set of sub-models with low-cardinality exogenous variables. These sub-models had precise marginals over the exogenous variables and therefore admitted efficient exact inference. The aggregated results were used to bound counterfactual probabilities in the original model. The approach was developed for Markovian models, where each exogenous variable affects only a single endogenous variable. In this paper, we investigate extending the methodology to \textit{semi-Markovian} SCMs, where exogenous variables may influence multiple endogenous variables. Such models are capable of representing confounding relationships that Markovian models cannot. We illustrate the challenges of this extension using a minimal example, which motivates a set of alternative solution strategies. These strategies are evaluated both theoretically and through a computational study.

Method: Constructed a transportation-specific harmful query dataset, developed a novel jailbreaking attack using image typography manipulation and multi-turn prompting, and proposed a multi-layered response filtering defense technique.

Result: Extensive experiments on state-of-the-art LVLMs (both open-source and closed-source) showed severe security vulnerabilities, with GPT-4 judgment and manual verification confirming the effectiveness of the proposed attack method compared to existing techniques.

Conclusion: The study highlights critical security risks in LVLMs integrated in ITS and demonstrates that image typography manipulation combined with multi-turn prompting creates powerful jailbreaking attacks that require robust defense mechanisms.

Abstract: Large Vision Language Models (LVLMs) demonstrate strong capabilities in multimodal reasoning and many real-world applications, such as visual question answering. However, LVLMs are highly vulnerable to jailbreaking attacks. This paper systematically analyzes the vulnerabilities of LVLMs integrated in Intelligent Transportation Systems (ITS) under carefully crafted jailbreaking attacks. First, we carefully construct a dataset with harmful queries relevant to transportation, following OpenAI’s prohibited categories to which the LVLMs should not respond. Second, we introduce a novel jailbreaking attack that exploits the vulnerabilities of LVLMs through image typography manipulation and multi-turn prompting. Third, we propose a multi-layered response filtering defense technique to prevent the model from generating inappropriate responses. We perform extensive experiments with the proposed attack and defense on the state-of-the-art LVLMs (both open-source and closed-source). To evaluate the attack method and defense technique, we use GPT-4’s judgment to determine the toxicity score of the generated responses, as well as manual verification. Further, we compare our proposed jailbreaking method with existing jailbreaking techniques and highlight severe security risks involved with jailbreaking attacks with image typography manipulation and multi-turn prompting in the LVLMs integrated in ITS.

Method: Proposes DataSage with three key features: external knowledge retrieval for enriched context, multi-role debating mechanism for diverse analytical perspectives, and multi-path reasoning for improved code and insight accuracy.

Result: Extensive experiments on InsightBench show DataSage consistently outperforms existing data insight agents across all difficulty levels.

Conclusion: DataSage provides an effective solution for automated data insight discovery by addressing key limitations of current LLM-driven agents.

Abstract: In today’s data-driven era, fully automated end-to-end data analytics, particularly insight discovery, is critical for discovering actionable insights that assist organizations in making effective decisions. With the rapid advancement of large language models (LLMs), LLM-driven agents have emerged as a promising paradigm for automating data analysis and insight discovery. However, existing data insight agents remain limited in several key aspects, often failing to deliver satisfactory results due to: (1) insufficient utilization of domain knowledge, (2) shallow analytical depth, and (3) error-prone code generation during insight generation. To address these issues, we propose DataSage, a novel multi-agent framework that incorporates three innovative features including external knowledge retrieval to enrich the analytical context, a multi-role debating mechanism to simulate diverse analytical perspectives and deepen analytical depth, and multi-path reasoning to improve the accuracy of the generated code and insights. Extensive experiments on InsightBench demonstrate that DataSage consistently outperforms existing data insight agents across all difficulty levels, offering an effective solution for automated data insight discovery.

Method: CORGI uses a two-step approach: builds/maintains a graph of grounded relations in a forward pass, then generates matches iteratively by working backward through the graph, eliminating traditional β-memory for partial matches.

Result: CORGI significantly outperforms RETE implementations from SOAR and OPS5 on combinatorial matching tasks, avoiding high-latency delays and memory overflows.

Conclusion: CORGI makes rule-based systems practical for real-time AI applications by providing predictable performance guarantees and eliminating worst-case matching patterns that can halt execution.

Abstract: Rule-based systems must solve complex matching problems within tight time constraints to be effective in real-time applications, such as planning and reactive control for AI agents, as well as low-latency relational database querying. Pattern-matching systems can encounter issues where exponential time and space are required to find matches for rules with many underconstrained variables, or which produce combinatorial intermediate partial matches (but are otherwise well-constrained). When online AI systems automatically generate rules from example-driven induction or code synthesis, they can easily produce worst-case matching patterns that slow or halt program execution by exceeding available memory. In our own work with cognitive systems that learn from example, we’ve found that aggressive forms of anti-unification-based generalization can easily produce these circumstances. To make these systems practical without hand-engineering constraints or succumbing to unpredictable failure modes, we introduce a new matching algorithm called CORGI (Collection-Oriented Relational Graph Iteration). Unlike RETE-based approaches, CORGI offers quadratic time and space guarantees for finding single satisficing matches, and the ability to iteratively stream subsequent matches without committing entire conflict sets to memory. CORGI differs from RETE in that it does not have a traditional $β$-memory for collecting partial matches. Instead, CORGI takes a two-step approach: a graph of grounded relations is built/maintained in a forward pass, and an iterator generates matches as needed by working backward through the graph. This approach eliminates the high-latency delays and memory overflows that can result from populating full conflict sets. In a performance evaluation, we demonstrate that CORGI significantly outperforms RETE implementations from SOAR and OPS5 on a simple combinatorial matching task.

Method: OSHA narratives are analyzed using GPT-4o to extract structured hazard reasoning, converted into object-level scene graphs with spatial relationships, which guide text-to-image diffusion models to generate compositionally accurate hazard scenes. A visual question answering (VQA) framework evaluates realism and semantic fidelity.

Result: The proposed VQA Graph Score outperforms CLIP and BLIP metrics across four state-of-the-art generative models based on entropy-based validation, confirming higher discriminative sensitivity for evaluating generated hazard images.

Conclusion: The framework successfully generates realistic workplace hazard images from textual accident reports, with the VQA-based evaluation metric providing superior assessment of semantic fidelity compared to existing methods.

Abstract: Training vision models to detect workplace hazards accurately requires realistic images of unsafe conditions that could lead to accidents. However, acquiring such datasets is difficult because capturing accident-triggering scenarios as they occur is nearly impossible. To overcome this limitation, this study presents a novel scene graph-guided generative AI framework that synthesizes photorealistic images of hazardous scenarios grounded in historical Occupational Safety and Health Administration (OSHA) accident reports. OSHA narratives are analyzed using GPT-4o to extract structured hazard reasoning, which is converted into object-level scene graphs capturing spatial and contextual relationships essential for understanding risk. These graphs guide a text-to-image diffusion model to generate compositionally accurate hazard scenes. To evaluate the realism and semantic fidelity of the generated data, a visual question answering (VQA) framework is introduced. Across four state-of-the-art generative models, the proposed VQA Graph Score outperforms CLIP and BLIP metrics based on entropy-based validation, confirming its higher discriminative sensitivity.

Method: Dual-case methodology: generative AI in secondary education and multi-agent system for symbolic music analysis using modular, scalable workflows.

Result: AI agents outperform traditional methods in interpretability and adaptability, enhancing musical pattern recognition and educational feedback.

Conclusion: Provides a unified framework bridging technical, pedagogical, and ethical considerations for responsible AI deployment in musicology and education.

Abstract: This paper presents an integrative review and experimental validation of artificial intelligence (AI) agents applied to music analysis and education. We synthesize the historical evolution from rule-based models to contemporary approaches involving deep learning, multi-agent architectures, and retrieval-augmented generation (RAG) frameworks. The pedagogical implications are evaluated through a dual-case methodology: (1) the use of generative AI platforms in secondary education to foster analytical and creative skills; (2) the design of a multiagent system for symbolic music analysis, enabling modular, scalable, and explainable workflows. Experimental results demonstrate that AI agents effectively enhance musical pattern recognition, compositional parameterization, and educational feedback, outperforming traditional automated methods in terms of interpretability and adaptability. The findings highlight key challenges concerning transparency, cultural bias, and the definition of hybrid evaluation metrics, emphasizing the need for responsible deployment of AI in educational environments. This research contributes to a unified framework that bridges technical, pedagogical, and ethical considerations, offering evidence-based guidance for the design and application of intelligent agents in computational musicology and music education.

Method: Two-stage approach: 1) Adaptive Layer Attention groups encoder layers into semantic blocks with multi-head attention fusion, 2) Multi-objective knowledge distillation aligns noisy student with clean teacher’s semantic and attention distributions.

Result: Significant reduction in hallucinations and word error rates on noisy speech benchmarks while preserving clean speech performance.

Conclusion: The combined ALA and KD approach provides a principled strategy to enhance Whisper’s reliability in real-world noisy conditions.

Abstract: The Whisper model, an open-source automatic speech recognition system, is widely adopted for its strong performance across multilingual and zero-shot settings. However, it frequently suffers from hallucination errors, especially under noisy acoustic conditions. Previous works to reduce hallucinations in Whisper-style ASR systems have primarily focused on audio preprocessing or post-processing of transcriptions to filter out erroneous content. However, modifications to the Whisper model itself remain largely unexplored to mitigate hallucinations directly. To address this challenge, we present a two-stage architecture that first enhances encoder robustness through Adaptive Layer Attention (ALA) and further suppresses hallucinations using a multi-objective knowledge distillation (KD) framework. In the first stage, ALA groups encoder layers into semantically coherent blocks via inter-layer correlation analysis. A learnable multi-head attention module then fuses these block representations, enabling the model to jointly exploit low- and high-level features for more robust encoding. In the second stage, our KD framework trains the student model on noisy audio to align its semantic and attention distributions with a teacher model processing clean inputs. Our experiments on noisy speech benchmarks show notable reductions in hallucinations and word error rates, while preserving performance on clean speech. Together, ALA and KD offer a principled strategy to improve Whisper’s reliability under real-world noisy conditions.

Method: ALEX uses hierarchical memory architecture to organize knowledge edits into semantic clusters, integrates Inferential Query Synthesis to bridge semantic gaps, and employs Dynamic Evidence Adjudication for two-stage retrieval.

Result: On MQUAKE benchmark, ALEX significantly improves multi-hop answer accuracy (MultiHop-ACC) and reasoning path reliability (HopWise-ACC), while reducing search space by over 80%.

Conclusion: ALEX presents a promising approach for building scalable, efficient, and accurate knowledge editing systems by addressing fundamental retrieval complexity challenges.

Abstract: The static nature of knowledge within Large Language Models (LLMs) makes it difficult for them to adapt to evolving information, rendering knowledge editing a critical task. However, existing methods struggle with challenges of scalability and retrieval efficiency, particularly when handling complex, multi-hop questions that require multi-step reasoning. To address these challenges, this paper introduces ALEX (A Light Editing-knowledge Extractor), a lightweight knowledge editing framework. The core innovation of ALEX is its hierarchical memory architecture, which organizes knowledge updates (edits) into semantic clusters. This design fundamentally reduces retrieval complexity from a linear O(N) to a highly scalable O(K+N/C). Furthermore, the framework integrates an Inferential Query Synthesis (IQS) module to bridge the semantic gap between queries and facts , and a Dynamic Evidence Adjudication (DEA) engine that executes an efficient two-stage retrieval process. Experiments on the MQUAKE benchmark demonstrate that ALEX significantly improves both the accuracy of multi-hop answers (MultiHop-ACC) and the reliability of reasoning paths (HopWise-ACC). It also reduces the required search space by over 80% , presenting a promising path toward building scalable, efficient, and accurate knowledge editing systems.

Method: Developed Syn-STARTS framework that uses Large Language Models (LLMs) to generate synthetic triage cases, then compared them against manually curated TRIAGE dataset using standard START triage categories.

Result: Generated triage cases were qualitatively indistinguishable from manually curated data, and LLM accuracy evaluation across green, yellow, red, and black categories showed high stability.

Conclusion: Synthetic data generation using LLMs shows strong potential for developing high-performance AI models in severe medical situations where real data is scarce.

Abstract: Triage is a critically important decision-making process in mass casualty incidents (MCIs) to maximize victim survival rates. While the role of AI in such situations is gaining attention for making optimal decisions within limited resources and time, its development and performance evaluation require benchmark datasets of sufficient quantity and quality. However, MCIs occur infrequently, and sufficient records are difficult to accumulate at the scene, making it challenging to collect large-scale realworld data for research use. Therefore, we developed Syn-STARTS, a framework that uses LLMs to generate triage cases, and verified its effectiveness. The results showed that the triage cases generated by Syn-STARTS were qualitatively indistinguishable from the TRIAGE open dataset generated by manual curation from training materials. Furthermore, when evaluating the LLM accuracy using hundreds of cases each from the green, yellow, red, and black categories defined by the standard triage method START, the results were found to be highly stable. This strongly indicates the possibility of synthetic data in developing high-performance AI models for severe and critical medical situations.

Method: Uses binary integer programming with constraints for coverage, time, difficulty windows, prerequisites, and anti-redundancy. Employs greedy selection for low-richness scenarios, gradient-based relaxation for rich repositories, and hybrid methods.

Result: Achieved full skill coverage for 1204 students within bounded watch time, reduced redundant coverage by ~12% compared to greedy approach, and maintained sufficiency across subgroups.

Conclusion: The system provides a tractable and auditable controller that closes the diagnostic-pedagogical loop and delivers equitable, load-aware personalization at classroom scale.

Abstract: Adaptive learning often diagnoses precisely yet intervenes weakly, yielding help that is mistimed or misaligned. This study presents evidence supporting an instructor-governed feedback loop that converts concept-level assessment evidence into vetted micro-interventions. The adaptive learning algorithm contains three safeguards: adequacy as a hard guarantee of gap closure, attention as a budgeted constraint for time and redundancy, and diversity as protection against overfitting to a single resource. We formalize intervention assignment as a binary integer program with constraints for coverage, time, difficulty windows informed by ability estimates, prerequisites encoded by a concept matrix, and anti-redundancy enforced through diversity. Greedy selection serves low-richness and tight-latency regimes, gradient-based relaxation serves rich repositories, and a hybrid method transitions along a richness-latency frontier. In simulation and in an introductory physics deployment with one thousand two hundred four students, both solvers achieved full skill coverage for essentially all learners within bounded watch time. The gradient-based method reduced redundant coverage by approximately twelve percentage points relative to greedy and harmonized difficulty across slates, while greedy delivered comparable adequacy with lower computational cost in scarce settings. Slack variables localized missing content and supported targeted curation, sustaining sufficiency across subgroups. The result is a tractable and auditable controller that closes the diagnostic-pedagogical loop and delivers equitable, load-aware personalization at classroom scale.

Method: A multi-agent framework with OrchestratorAgent, SemanticParserAgent, PrimaryLayoutAgent, TemplateRetrievalAgent, and RecursiveComponentAgent that collaboratively parse user descriptions and generate hierarchical layouts.

Result: Experimental results on the RICO dataset show that APD-agents achieve state-of-the-art performance in automated page design.

Conclusion: The framework successfully leverages LLM-driven multi-agent collaboration to automate mobile app page design, addressing the challenges of manual design processes.

Abstract: Layout design is a crucial step in developing mobile app pages. However, crafting satisfactory designs is time-intensive for designers: they need to consider which controls and content to present on the page, and then repeatedly adjust their size, position, and style for better aesthetics and structure. Although many design software can now help to perform these repetitive tasks, extensive training is needed to use them effectively. Moreover, collaborative design across app pages demands extra time to align standards and ensure consistent styling. In this work, we propose APD-agents, a large language model (LLM) driven multi-agent framework for automated page design in mobile applications. Our framework contains OrchestratorAgent, SemanticParserAgent, PrimaryLayoutAgent, TemplateRetrievalAgent, and RecursiveComponentAgent. Upon receiving the user’s description of the page, the OrchestratorAgent can dynamically can direct other agents to accomplish users’ design task. To be specific, the SemanticParserAgent is responsible for converting users’ descriptions of page content into structured data. The PrimaryLayoutAgent can generate an initial coarse-grained layout of this page. The TemplateRetrievalAgent can fetch semantically relevant few-shot examples and enhance the quality of layout generation. Besides, a RecursiveComponentAgent can be used to decide how to recursively generate all the fine-grained sub-elements it contains for each element in the layout. Our work fully leverages the automatic collaboration capabilities of large-model-driven multi-agent systems. Experimental results on the RICO dataset show that our APD-agents achieve state-of-the-art performance.

Method: PRISM integrates three components: refined system prompting for precise instructions, in-context learning with relevant examples, and a lightweight multi-agent system for coordinated scoring behavior.

Result: Achieves NDCG@5 of 0.71818 on restricted validation split, demonstrating feasibility and robustness for production-scale financial retrieval.

Conclusion: PRISM’s modular, inference-only design makes it practical for real-world financial retrieval applications without requiring model training.

Abstract: With the rapid progress of large language models (LLMs), financial information retrieval has become a critical industrial application. Extracting task-relevant information from lengthy financial filings is essential for both operational and analytical decision-making. The FinAgentBench dataset formalizes this problem through two tasks: document ranking and chunk ranking. We present PRISM, a training-free framework that integrates refined system prompting, in-context learning (ICL), and a lightweight multi-agent system. Each component is examined extensively to reveal their synergies: prompt engineering provides precise task instructions, ICL supplies semantically relevant few-shot examples, and the multi-agent system models coordinated scoring behaviour. Our best configuration achieves an NDCG@5 of 0.71818 on the restricted validation split. We further demonstrate that PRISM is feasible and robust for production-scale financial retrieval. Its modular, inference-only design makes it practical for real-world use cases. The source code is released at https://bit.ly/prism-ailens.

Method: Proposes MAKGED framework using multiple LLMs in collaborative setting, concatenating fine-grained bidirectional subgraph embeddings with LLM-based query embeddings to create four specialized agents that engage in multi-round discussions.

Result: Extensive experiments on FB15K and WN18RR show MAKGED outperforms state-of-the-art methods, enhancing accuracy and robustness of KG evaluation.

Conclusion: The framework enables training specialized agents using domain-specific knowledge graphs for error detection, highlighting significant industrial application potential.

Abstract: Knowledge graphs are widely used in industrial applications, making error detection crucial for ensuring the reliability of downstream applications. Existing error detection methods often fail to effectively utilize fine-grained subgraph information and rely solely on fixed graph structures, while also lacking transparency in their decision-making processes, which results in suboptimal detection performance. In this paper, we propose a novel Multi-Agent framework for Knowledge Graph Error Detection (MAKGED) that utilizes multiple large language models (LLMs) in a collaborative setting. By concatenating fine-grained, bidirectional subgraph embeddings with LLM-based query embeddings during training, our framework integrates these representations to produce four specialized agents. These agents utilize subgraph information from different dimensions to engage in multi-round discussions, thereby improving error detection accuracy and ensuring a transparent decision-making process. Extensive experiments on FB15K and WN18RR demonstrate that MAKGED outperforms state-of-the-art methods, enhancing the accuracy and robustness of KG evaluation. For specific industrial scenarios, our framework can facilitate the training of specialized agents using domain-specific knowledge graphs for error detection, which highlights the potential industrial application value of our framework. Our code and datasets are available at https://github.com/kse-ElEvEn/MAKGED.

Method: A three-module framework: Runner (lightweight transformer expert for efficient navigation), Ruminator (multimodal LLM with chain-of-thought reasoning), and Regulator (monitors progress and controls thinking modes).

Result: Outperforms state-of-the-art methods by 3.28% in SPL and 3.30% in RGSPL on REVERIE benchmark.

Conclusion: The dual-process framework effectively handles challenging VLN tasks by harmoniously integrating LLM generalization with domain-specific expertise.

Abstract: Vision-and-Language Navigation (VLN) requires an agent to dynamically explore complex 3D environments following human instructions. Recent research underscores the potential of harnessing large language models (LLMs) for VLN, given their commonsense knowledge and general reasoning capabilities. Despite their strengths, a substantial gap in task completion performance persists between LLM-based approaches and domain experts, as LLMs inherently struggle to comprehend real-world spatial correlations precisely. Additionally, introducing LLMs is accompanied with substantial computational cost and inference latency. To address these issues, we propose a novel dual-process thinking framework dubbed R3, integrating LLMs’ generalization capabilities with VLN-specific expertise in a zero-shot manner. The framework comprises three core modules: Runner, Ruminator, and Regulator. The Runner is a lightweight transformer-based expert model that ensures efficient and accurate navigation under regular circumstances. The Ruminator employs a powerful multimodal LLM as the backbone and adopts chain-of-thought (CoT) prompting to elicit structured reasoning. The Regulator monitors the navigation progress and controls the appropriate thinking mode according to three criteria, integrating Runner and Ruminator harmoniously. Experimental results illustrate that R3 significantly outperforms other state-of-the-art methods, exceeding 3.28% and 3.30% in SPL and RGSPL respectively on the REVERIE benchmark. This pronounced enhancement highlights the effectiveness of our method in handling challenging VLN tasks.

Method: MI9 integrates six components: agency-risk index, agent-semantic telemetry capture, continuous authorization monitoring, FSM-based conformance engines, goal-conditioned drift detection, and graduated containment strategies.

Result: The framework enables systematic, safe deployment of agentic systems in production environments, providing comprehensive coverage of governance challenges that existing approaches fail to address.

Conclusion: MI9 establishes the technical foundation for comprehensive agentic AI oversight at scale, operating transparently across heterogeneous agent architectures.

Abstract: Agentic AI systems capable of reasoning, planning, and executing actions present fundamentally distinct governance challenges compared to traditional AI models. Unlike conventional AI, these systems exhibit emergent and unexpected behaviors during runtime, introducing novel agent-related risks that cannot be fully anticipated through pre-deployment governance alone. To address this critical gap, we introduce MI9, the first fully integrated runtime governance framework designed specifically for safety and alignment of agentic AI systems. MI9 introduces real-time controls through six integrated components: agency-risk index, agent-semantic telemetry capture, continuous authorization monitoring, Finite-State-Machine (FSM)-based conformance engines, goal-conditioned drift detection, and graduated containment strategies. Operating transparently across heterogeneous agent architectures, MI9 enables the systematic, safe, and responsible deployment of agentic systems in production environments where conventional governance approaches fall short, providing the foundational infrastructure for safe agentic AI deployment at scale. Detailed analysis through a diverse set of scenarios demonstrates MI9’s systematic coverage of governance challenges that existing approaches fail to address, establishing the technical foundation for comprehensive agentic AI oversight.

Method: Proposed CLEAR framework (Cost, Latency, Efficacy, Assurance, Reliability) for holistic enterprise evaluation. Evaluated 6 leading agents on 300 enterprise tasks and conducted expert evaluation (N=15).

Result: Accuracy-optimized agents are 4.4-10.8x more expensive than cost-aware alternatives with comparable performance. CLEAR better predicts production success (ρ=0.83) vs accuracy-only evaluation (ρ=0.41).

Conclusion: CLEAR provides a comprehensive evaluation framework that better aligns with enterprise deployment needs, addressing cost, reliability, and multidimensional metrics beyond just accuracy.

Abstract: Current agentic AI benchmarks predominantly evaluate task completion accuracy, while overlooking critical enterprise requirements such as cost-efficiency, reliability, and operational stability. Through systematic analysis of 12 main benchmarks and empirical evaluation of state-of-the-art agents, we identify three fundamental limitations: (1) absence of cost-controlled evaluation leading to 50x cost variations for similar precision, (2) inadequate reliability assessment where agent performance drops from 60% (single run) to 25% (8-run consistency), and (3) missing multidimensional metrics for security, latency, and policy compliance. We propose \textbf{CLEAR} (Cost, Latency, Efficacy, Assurance, Reliability), a holistic evaluation framework specifically designed for enterprise deployment. Evaluation of six leading agents on 300 enterprise tasks demonstrates that optimizing for accuracy alone yields agents 4.4-10.8x more expensive than cost-aware alternatives with comparable performance. Expert evaluation (N=15) confirms that CLEAR better predicts production success (correlation $ρ=0.83$) compared to accuracy-only evaluation ($ρ=0.41$).

Method: Uses learnable protocols, representation mapping, and KL-budget constraints to enable controlled co-evolution between humans and AI systems.

Result: Achieved 85.5% success rate in collaborative navigation (vs 70.3% baseline), 230% better mutual adaptation, 332% better protocol convergence. Emergent protocols outperformed handcrafted ones by 84%, with 23% better safety and out-of-distribution robustness.

Conclusion: Demonstrates optimal collaboration exists at the intersection of human and AI capabilities, validating the shift from single-directional to co-alignment paradigms with 46% synergy improvement.

Abstract: Current AI alignment through RLHF follows a single directional paradigm that AI conforms to human preferences while treating human cognition as fixed. We propose a shift to co-alignment through Bidirectional Cognitive Alignment (BiCA), where humans and AI mutually adapt. BiCA uses learnable protocols, representation mapping, and KL-budget constraints for controlled co-evolution. In collaborative navigation, BiCA achieved 85.5% success versus 70.3% baseline, with 230% better mutual adaptation and 332% better protocol convergence. Emergent protocols outperformed handcrafted ones by 84%, while bidirectional adaptation unexpectedly improved safety (+23% out-of-distribution robustness). The 46% synergy improvement demonstrates optimal collaboration exists at the intersection, not union, of human and AI capabilities, validating the shift from single-directional to co-alignment paradigms.

Method: Proposed HFL-FlowLLM framework that applies large language models to network traffic flow classification in heterogeneous federated learning environments.

Result: 13% improvement in average F1 score over state-of-the-art methods, up to 5% improvement in F1 score with more clients, and 87% reduction in training costs compared to existing LLM federated learning frameworks.

Conclusion: HFL-FlowLLM demonstrates strong potential and practical value for network security in modern communication networks with compelling performance and robustness.

Abstract: In modern communication networks driven by 5G and the Internet of Things (IoT), effective network traffic flow classification is crucial for Quality of Service (QoS) management and security. Traditional centralized machine learning struggles with the distributed data and privacy concerns in these heterogeneous environments, while existing federated learning approaches suffer from high costs and poor generalization. To address these challenges, we propose HFL-FlowLLM, which to our knowledge is the first framework to apply large language models to network traffic flow classification in heterogeneous federated learning. Compared to state-of-the-art heterogeneous federated learning methods for network traffic flow classification, the proposed approach improves the average F1 score by approximately 13%, demonstrating compelling performance and strong robustness. When compared to existing large language models federated learning frameworks, as the number of clients participating in each training round increases, the proposed method achieves up to a 5% improvement in average F1 score while reducing the training costs by about 87%. These findings prove the potential and practical value of HFL-FlowLLM in modern communication networks security.

Method: Evaluated multiple LLMs (GPT-4.1, Claude-3.7 Sonnet with/without Extended Thinking, GPT-5) on three chronological tasks: basic ordering, conditional sorting (filter then order), and anachronism detection, across different reasoning-effort settings.

Result: Exact match rates drop sharply with longer sequences, though rank correlations remain high. Conditional sorting failures mainly occur during filtering. GPT-5 with medium/high reasoning effort achieved flawless ordering and perfect conditional sorting, while low-effort settings degraded with longer lists.

Conclusion: Current LLMs have limitations in chronological reasoning, but explicit reasoning allocation helps significantly. These findings are important for real-time financial applications of LLMs.

Abstract: Large language models (LLMs) are increasingly used in finance and economics, where prompt-based attempts against look-ahead bias implicitly assume that models understand chronology. We test this fundamental question with a series of chronological ordering tasks with increasing complexities over facts the model already knows from pre-training. Our tasks cover (1) chronological ordering, (2) conditional sorting (filter, then order), and (3) anachronism detection. We evaluate GPT-4.1, Claude-3.7 Sonnet, with and without Extended Thinking (ET), and GPT-5 across multiple reasoning-effort settings. Across models, Exact match rate drops sharply as sequences lengthen even while rank correlations stay high as LLMs largely preserve local order but struggle to maintain a single globally consistent timeline. In conditional sorting, most failures stem from the filtering step rather than the ordering step, but GPT-5 and Claude-3.7 Sonnet with Extended Thinking outshine normal models significantly. Lastly, anachronism detection is found to be the easiest task for the LLMs but performance still declines with increasingly overlapping timelines or entities. Overall, our main contribution is showing that allocating explicit reasoning budget helps with chronological ordering with GPT-5 at medium/high reasoning effort achieving flawless ordering at all lengths and perfect conditional sorting (both self-filtered and given-subset), whereas low/minimal effort degrades with longer lists, mirroring earlier models. Our findings delineate limits of current LLMs on chronological tasks, providing insights into task complexity, and demonstrate scenarios in which reasoning helps. These patterns are important for the real-time application of LLMs in finance. We release all code and evaluation templates to support full reproducibility.

Method: 1) Equip LLM with IoT domain knowledge via continual pre-training and multi-task fine-tuning; 2) Use direct preference optimization to align with user preferences; 3) Implement confidence-based exposure control to avoid low-quality recommendations.

Result: DevPiolt outperforms baselines by 69.5% average improvement across all metrics. In real deployment (Xiaomi Home app), it increased unique visitor device coverage by 21.6% and page view acceptance rates by 29.1% for 255,000 users.

Conclusion: DevPiolt successfully addresses IoT operation recommendation challenges and demonstrates significant improvements in both offline experiments and real-world deployment.

Abstract: Operation recommendation for IoT devices refers to generating personalized device operations for users based on their context, such as historical operations, environment information, and device status. This task is crucial for enhancing user satisfaction and corporate profits. Existing recommendation models struggle with complex operation logic, diverse user preferences, and sensitive to suboptimal suggestions, limiting their applicability to IoT device operations. To address these issues, we propose DevPiolt, a LLM-based recommendation model for IoT device operations. Specifically, we first equip the LLM with fundamental domain knowledge of IoT operations via continual pre-training and multi-task fine-tuning. Then, we employ direct preference optimization to align the fine-tuned LLM with specific user preferences. Finally, we design a confidence-based exposure control mechanism to avoid negative user experiences from low-quality recommendations. Extensive experiments show that DevPiolt significantly outperforms baselines on all datasets, with an average improvement of 69.5% across all metrics. DevPiolt has been practically deployed in Xiaomi Home app for one quarter, providing daily operation recommendations to 255,000 users. Online experiment results indicate a 21.6% increase in unique visitor device coverage and a 29.1% increase in page view acceptance rates.

Method: Sliding-window approach with regional representations combining listing features, urban accessibility, and human mobility. Tabular data converted to LLM prompts for embeddings, then fed into RNN/LSTM/Transformer models.

Result: 48% reduction in average RMSE and MAE compared to traditional baselines on Seoul Airbnb dataset. Framework successfully forecasts Revenue, Reservation Days, and Number of Reservations 1-3 months ahead.

Conclusion: Method improves forecasting accuracy and provides practical insights for detecting oversupplied regions and supporting data-driven urban policy decisions.

Abstract: The expansion of short-term rental platforms, such as Airbnb, has significantly disrupted local housing markets, often leading to increased rental prices and housing affordability issues. Accurately forecasting regional Airbnb market trends can thus offer critical insights for policymakers and urban planners aiming to mitigate these impacts. This study proposes a novel time-series forecasting framework to predict three key Airbnb indicators – Revenue, Reservation Days, and Number of Reservations – at the regional level. Using a sliding-window approach, the model forecasts trends 1 to 3 months ahead. Unlike prior studies that focus on individual listings at fixed time points, our approach constructs regional representations by integrating listing features with external contextual factors such as urban accessibility and human mobility. We convert structured tabular data into prompt-based inputs for a Large Language Model (LLM), producing comprehensive regional embeddings. These embeddings are then fed into advanced time-series models (RNN, LSTM, Transformer) to better capture complex spatio-temporal dynamics. Experiments on Seoul’s Airbnb dataset show that our method reduces both average RMSE and MAE by approximately 48% compared to conventional baselines, including traditional statistical and machine learning models. Our framework not only improves forecasting accuracy but also offers practical insights for detecting oversupplied regions and supporting data-driven urban policy decisions.

Method: Follows “Retrieve-Prioritize-Reason” paradigm: retrieves query subgraph, introduces path prioritization mechanism using semantic-aware path priority function (considering accumulative and estimated future costs), and generates responses via dual-phase training (task-specific instruction tuning and path-wise preference alignment).

Result: Extensive experiments show PathMind consistently outperforms competitive baselines, particularly on complex reasoning tasks with fewer input tokens, by identifying essential reasoning paths.

Conclusion: PathMind enhances faithful and interpretable reasoning by selectively guiding LLMs with important reasoning paths, addressing both noise and efficiency limitations of existing LLM-based KGR methods.

Abstract: Knowledge graph reasoning (KGR) is the task of inferring new knowledge by performing logical deductions on knowledge graphs. Recently, large language models (LLMs) have demonstrated remarkable performance in complex reasoning tasks. Despite promising success, current LLM-based KGR methods still face two critical limitations. First, existing methods often extract reasoning paths indiscriminately, without assessing their different importance, which may introduce irrelevant noise that misleads LLMs. Second, while many methods leverage LLMs to dynamically explore potential reasoning paths, they require high retrieval demands and frequent LLM calls. To address these limitations, we propose PathMind, a novel framework designed to enhance faithful and interpretable reasoning by selectively guiding LLMs with important reasoning paths. Specifically, PathMind follows a “Retrieve-Prioritize-Reason” paradigm. First, it retrieves a query subgraph from KG through the retrieval module. Next, it introduces a path prioritization mechanism that identifies important reasoning paths using a semantic-aware path priority function, which simultaneously considers the accumulative cost and the estimated future cost for reaching the target. Finally, PathMind generates accurate and logically consistent responses via a dual-phase training strategy, including task-specific instruction tuning and path-wise preference alignment. Extensive experiments on benchmark datasets demonstrate that PathMind consistently outperforms competitive baselines, particularly on complex reasoning tasks with fewer input tokens, by identifying essential reasoning paths.

Method: Systematically rephrased and perturbed well-known CSPLib problems to preserve structure while modifying context and adding misleading elements, then compared models generated by three representative LLMs across original and modified descriptions.

Result: LLMs can produce syntactically valid and semantically plausible models, but performance drops sharply under contextual and linguistic variation, revealing shallow understanding and sensitivity to wording.

Conclusion: LLMs demonstrate limited genuine reasoning capabilities for constraint programming, with their performance heavily dependent on surface-level linguistic patterns rather than deep structural understanding.

Abstract: One of the long-standing goals in optimisation and constraint programming is to describe a problem in natural language and automatically obtain an executable, efficient model. Large language models appear to bring this vision closer, showing impressive results in automatically generating models for classical benchmarks. However, much of this apparent success may derive from data contamination rather than genuine reasoning: many standard CP problems are likely included in the training data of these models. To examine this hypothesis, we systematically rephrased and perturbed a set of well-known CSPLib problems to preserve their structure while modifying their context and introducing misleading elements. We then compared the models produced by three representative LLMs across original and modified descriptions. Our qualitative analysis shows that while LLMs can produce syntactically valid and semantically plausible models, their performance drops sharply under contextual and linguistic variation, revealing shallow understanding and sensitivity to wording.

Method: Collected alignment data from US and German participants (N=1,095, 27,375 ratings) across five dimensions, fine-tuned multiple LLMs using preferences from different social groups while varying rating scales, disagreement handling methods, and optimization techniques.

Result: Systematic demographic effects found: male participants rated responses 18% less toxic than females; conservative and Black participants rated responses 27.9% and 44% more emotionally aware than liberal and White participants. Technical choices showed strong effects: preserving disagreement achieved 53% greater toxicity reduction than majority voting, 5-point scales yielded 22% more reduction than binary formats, and DPO outperformed GRPO.

Conclusion: These findings represent a preliminary step in answering how alignment should balance expert-driven and user-driven signals to ensure both safety and fair representation across diverse social groups.

Abstract: Although large language models (LLMs) are increasingly trained using human feedback for safety and alignment with human values, alignment decisions often overlook human social diversity. This study examines how incorporating pluralistic values affects LLM behavior by systematically evaluating demographic variation and design parameters in the alignment pipeline. We collected alignment data from US and German participants (N = 1,095, 27,375 ratings) who rated LLM responses across five dimensions: Toxicity, Emotional Awareness (EA), Sensitivity, Stereotypical Bias, and Helpfulness. We fine-tuned multiple Large Language Models and Large Reasoning Models using preferences from different social groups while varying rating scales, disagreement handling methods, and optimization techniques. The results revealed systematic demographic effects: male participants rated responses 18% less toxic than female participants; conservative and Black participants rated responses 27.9% and 44% more emotionally aware than liberal and White participants, respectively. Models fine-tuned on group-specific preferences exhibited distinct behaviors. Technical design choices showed strong effects: the preservation of rater disagreement achieved roughly 53% greater toxicity reduction than majority voting, and 5-point scales yielded about 22% more reduction than binary formats; and Direct Preference Optimization (DPO) consistently outperformed Group Relative Policy Optimization (GRPO) in multi-value optimization. These findings represent a preliminary step in answering a critical question: How should alignment balance expert-driven and user-driven signals to ensure both safety and fair representation?

Method: Couples transformer-based perceptual front-end with GNN relational reasoning for probabilistic symbolic state extraction, combined with uncertainty-aware symbolic planner that actively gathers information when confidence is low.

Result: Achieved 94%/90%/88% success on robotic manipulation benchmarks (90.7% average), exceeding POMDP baselines by 10-14 points while planning within 15ms. Translator processed 10,047 scenes with F1=0.68 for probabilistic predicates.

Conclusion: The framework provides a principled connection between perception and planning with calibrated uncertainty, establishing quantitative links between uncertainty and planning convergence with theoretical guarantees validated empirically.

Abstract: Bridging continuous perceptual signals and discrete symbolic reasoning is a fundamental challenge in AI systems that must operate under uncertainty. We present a neuro-symbolic framework that explicitly models and propagates uncertainty from perception to planning, providing a principled connection between these two abstraction levels. Our approach couples a transformer-based perceptual front-end with graph neural network (GNN) relational reasoning to extract probabilistic symbolic states from visual observations, and an uncertainty-aware symbolic planner that actively gathers information when confidence is low. We demonstrate the framework’s effectiveness on tabletop robotic manipulation as a concrete application: the translator processes 10,047 PyBullet-generated scenes (3–10 objects) and outputs probabilistic predicates with calibrated confidences (overall F1=0.68). When embedded in the planner, the system achieves 94%/90%/88% success on Simple Stack, Deep Stack, and Clear+Stack benchmarks (90.7% average), exceeding the strongest POMDP baseline by 10–14 points while planning within 15,ms. A probabilistic graphical-model analysis establishes a quantitative link between calibrated uncertainty and planning convergence, providing theoretical guarantees that are validated empirically. The framework is general-purpose and can be applied to any domain requiring uncertainty-aware reasoning from perceptual input to symbolic planning.

Method: Model lecture content as metric-measure space, align candidate KGs using Fused Gromov-Wasserstein couplings to quantify semantic distortion, and apply refinement operators (add, merge, split, remove, rewire) to minimize rate-distortion Lagrangian.

Result: Prototype applied to data science lectures shows MCQs generated from refined KGs consistently surpass those from raw notes on fifteen quality criteria, with interpretable RD curves.

Conclusion: Establishes a principled foundation for information-theoretic KG optimization in personalized and AI-assisted education.

Abstract: Task-oriented knowledge graphs (KGs) enable AI-powered learning assistant systems to automatically generate high-quality multiple-choice questions (MCQs). Yet converting unstructured educational materials, such as lecture notes and slides, into KGs that capture key pedagogical content remains difficult. We propose a framework for knowledge graph construction and refinement grounded in rate-distortion (RD) theory and optimal transport geometry. In the framework, lecture content is modeled as a metric-measure space, capturing semantic and relational structure, while candidate KGs are aligned using Fused Gromov-Wasserstein (FGW) couplings to quantify semantic distortion. The rate term, expressed via the size of KG, reflects complexity and compactness. Refinement operators (add, merge, split, remove, rewire) minimize the rate-distortion Lagrangian, yielding compact, information-preserving KGs. Our prototype applied to data science lectures yields interpretable RD curves and shows that MCQs generated from refined KGs consistently surpass those from raw notes on fifteen quality criteria. This study establishes a principled foundation for information-theoretic KG optimization in personalized and AI-assisted education.

Method: Constructs a directed graph from historical agent trajectories where nodes represent tools and edges capture transition probabilities, modeling tool usage inertia. Integrates parameter-level information for refined tool input generation.

Result: Reduces inference costs by up to 30% while maintaining competitive task completion rates across diverse agent tasks.

Conclusion: AutoTool demonstrates that integrating statistical structure into LLM agent design enables greater efficiency without sacrificing performance, offering a practical enhancement for inference-heavy frameworks.

Abstract: Large Language Model (LLM) agents have emerged as powerful tools for automating complex tasks by leveraging the reasoning and decision-making abilities of LLMs. However, a major bottleneck in current agent frameworks lies in the high inference cost of tool selection, especially in approaches like ReAct that repeatedly invoke the LLM to determine which tool to use at each step. In this work, we propose AutoTool, a novel graph-based framework that bypasses repeated LLM inference by exploiting a key empirical observation: tool usage inertia - the tendency of tool invocations to follow predictable sequential patterns. AutoTool constructs a directed graph from historical agent trajectories, where nodes represent tools and edges capture transition probabilities, effectively modeling the inertia in tool selection. It further integrates parameter-level information to refine tool input generation. By traversing this structured representation, AutoTool efficiently selects tools and their parameters with minimal reliance on LLM inference. Extensive experiments across diverse agent tasks demonstrate that AutoTool reduces inference costs by up to 30% while maintaining competitive task completion rates, offering a practical and scalable enhancement for inference-heavy frameworks. Our work highlights the promise of integrating statistical structure into LLM agent design for greater efficiency without sacrificing performance.

Method: SkillGen constructs an action-centric domain-level graph from sampled trajectories, identifies high-utility actions via temporal-difference credit assignment, and retrieves step-wise skills to generate fine-grained, context-aware prompts.

Result: Experiments on ALFWorld, BabyAI, and ScienceWorld using both open-source and proprietary LLMs show that SkillGen achieves consistent gains, improving progress rate by 5.9%-16.5% on average across models.

Conclusion: Focusing on high-utility segments supports task identifiability and enables more effective ICL prompt design for sequential decision-making with LLMs.

Abstract: Large language models (LLMs) are increasingly applied to sequential decision-making through in-context learning (ICL), yet their effectiveness is highly sensitive to prompt quality. Effective prompts should meet three principles: focus on decision-critical information, provide step-level granularity, and minimize reliance on expert annotations through label efficiency. However, existing ICL methods often fail to satisfy all three criteria simultaneously. Motivated by these challenges, we introduce SkillGen, a skill-based ICL framework for structured sequential reasoning. It constructs an action-centric, domain-level graph from sampled trajectories, identifies high-utility actions via temporal-difference credit assignment, and retrieves step-wise skills to generate fine-grained, context-aware prompts. We further present a theoretical analysis showing that focusing on high-utility segments supports task identifiability and informs more effective ICL prompt design. Experiments on ALFWorld, BabyAI, and ScienceWorld, using both open-source and proprietary LLMs, show that SkillGen achieves consistent gains, improving progress rate by 5.9%-16.5% on average across models.

Method: Uses Heterogeneous-Agent Proximal Policy Optimization (HAPPO) with decentralized actor policies and centralized critic, trained in a physics-informed OpenDSS environment with differentiable penalty signals for operational constraints.

Result: HAPPO achieves faster convergence, higher restored power, and smoother multi-seed training than DQN, PPO, MAES, MAGDPG, MADQN, Mean-Field RL, and QMIX on IEEE 123-bus and IEEE 8500-node systems.

Conclusion: Incorporating microgrid-level heterogeneity within the HARL framework yields a scalable, stable, and constraint-aware solution for complex power distribution system restoration.

Abstract: Restoring power distribution systems (PDS) after large-scale outages requires sequential switching operations that reconfigure feeder topology and coordinate distributed energy resources (DERs) under nonlinear constraints such as power balance, voltage limits, and thermal ratings. These challenges make conventional optimization and value-based RL approaches computationally inefficient and difficult to scale. This paper applies a Heterogeneous-Agent Reinforcement Learning (HARL) framework, instantiated through Heterogeneous-Agent Proximal Policy Optimization (HAPPO), to enable coordinated restoration across interconnected microgrids. Each agent controls a distinct microgrid with different loads, DER capacities, and switch counts, introducing practical structural heterogeneity. Decentralized actor policies are trained with a centralized critic to compute advantage values for stable on-policy updates. A physics-informed OpenDSS environment provides full power flow feedback and enforces operational limits via differentiable penalty signals rather than invalid action masking. The total DER generation is capped at 2400 kW, and each microgrid must satisfy local supply-demand feasibility. Experiments on the IEEE 123-bus and IEEE 8500-node systems show that HAPPO achieves faster convergence, higher restored power, and smoother multi-seed training than DQN, PPO, MAES, MAGDPG, MADQN, Mean-Field RL, and QMIX. Results demonstrate that incorporating microgrid-level heterogeneity within the HARL framework yields a scalable, stable, and constraint-aware solution for complex PDS restoration.

Method: Created GraphInstruct benchmark with diverse graph generation pipelines and detailed reasoning steps, then developed GraphSolver via instruction-tuning and GraphSolver+ using label-mask training strategy with masked supervision on intermediate reasoning tokens.

Result: GraphSolver and GraphSolver+ demonstrate superior graph understanding and reasoning capabilities compared to other open-sourced LLMs, with GraphSolver+ showing enhanced multi-step reasoning through the label-mask approach.

Conclusion: GraphInstruct facilitates research on applying LLMs to graph-structured data, and the proposed methods successfully enhance LLMs’ graph understanding and reasoning abilities, with publicly released code and data.

Abstract: Improving the general capabilities of large language models (LLMs) is an active research topic. As a common data structure in many real-world domains, understanding graph data is a crucial part of advancing general intelligence. To this end, we propose a dynamic benchmark named GraphInstruct in this paper, which comprehensively includes 21 classical graph reasoning tasks, providing diverse graph generation pipelines and detailed intermediate reasoning steps for each sample. Based on GraphInstruct, we develop GraphSolver via efficient instruction-tuning, which demonstrates prominent graph understanding capability compared to other open-sourced LLMs. To further endow LLMs with multi-step graph reasoning capability, we propose a label-mask training strategy and build GraphSolver+, which leverages masked supervision on intermediate reasoning tokens to emphasize crucial node-identification signals. As one of the pioneering efforts to enhance the graph understanding and reasoning abilities of LLMs, extensive experiments have demonstrated the superiority of GraphSolver and GraphSolver+ over other LLMs. We sincerely hope GraphInstruct will facilitate further research on applying LLMs to graph-structured data. Our code and data are released publicly at: https://github.com/CGCL-codes/GraphInstruct.

Method: A three-step framework: 1) Break discharge summaries into small text chunks, 2) Use contextual BM25 and semantic search to identify chunks likely containing clinical events, 3) Employ Llama-3.1-8B with carefully designed prompts to extract or infer temporal information.

Result: Created MIMIC-IV-Ext-22MCTS dataset with 22.5M clinical time series events. BERT models fine-tuned on this dataset achieved 10% accuracy improvement on medical QA and 3% improvement on clinical trial matching compared to classic BERT.

Conclusion: The proposed framework successfully extracts structured temporal clinical data from unstructured medical notes, producing a valuable dataset that significantly enhances healthcare applications when used for model fine-tuning.

Abstract: A crucial component for clinical risk prediction is developing a reliable prediction model is collecting high-quality time series clinical events. In this work, we release such a dataset that consists of 22,588,586 Clinical Time Series events, which we term MIMIC-\RNum{4}-Ext-22MCTS. Our source data are discharge summaries selected from the well-known yet unstructured MIMIC-IV-Note \cite{Johnson2023-pg}. The general-purpose MIMIC-IV-Note pose specific challenges for our work: it turns out that the discharge summaries are too lengthy for typical natural language models to process, and the clinical events of interest often are not accompanied with explicit timestamps. Therefore, we propose a new framework that works as follows: 1) we break each discharge summary into manageably small text chunks; 2) we apply contextual BM25 and contextual semantic search to retrieve chunks that have a high potential of containing clinical events; and 3) we carefully design prompts to teach the recently released Llama-3.1-8B \cite{touvron2023llama} model to identify or infer temporal information of the chunks. The obtained dataset is informative and transparent that standard models fine-tuned on the dataset achieves significant improvements in healthcare applications. In particular, the BERT model fine-tuned based on our dataset achieves 10% improvement in accuracy on medical question answering task, and 3% improvement in clinical trial matching task compared with the classic BERT. The dataset is available at https://physionet.org/content/mimic-iv-ext-22mcts/1.0.0. The codebase is released at https://github.com/JingWang-RU/MIMIC-IV-Ext-22MCTS-Temporal-Clinical-Time-Series-Dataset.

Method: Investigated supervised fine-tuning (SFT), knowledge distillation (KD), reinforcement learning (RL), and their hybrid combinations to improve 0.5B SRLMs.

Result: Identified effective methodologies to bridge performance gap between small and large models, with insights into optimal training pipelines for 0.5B architectures.

Conclusion: Provides actionable recommendations for maximizing reasoning capabilities of 0.5B models through extensive experimental validation and analysis.

Abstract: The ongoing evolution of language models has led to the development of large-scale architectures that demonstrate exceptional performance across a wide range of tasks. However, these models come with significant computational and energy demands, as well as potential privacy implications. In this context, Small Reasoning Language Models (SRLMs) with approximately 0.5 billion parameters present a compelling alternative due to their remarkable computational efficiency and cost-effectiveness, particularly in resource-constrained environments. Despite these advantages, the limited capacity of 0.5 billion parameter models poses challenges in handling complex tasks such as mathematical reasoning. This research investigates various training strategies, including supervised fine-tuning (SFT), knowledge distillation (KD), and reinforcement learning (RL), as well as their hybrid implementations, to enhance the performance of 0.5B SRLMs. We analyze effective methodologies to bridge the performance gap between SRLMS and larger models and present insights into optimal training pipelines tailored for these smaller architectures. Through extensive experimental validation and analysis, our work aims to provide actionable recommendations for maximizing the reasoning capabilities of 0.5B models.

Method: Proposed a training framework using a large-scale dataset of 5M board-move pairs with expert annotations and engine evaluations, training Xiangqi-R1 in a multi-stage manner.

Result: Xiangqi-R1 achieved 18% improvement in move legality and 22% boost in analysis accuracy compared to general-purpose LLMs, which struggled with these tasks.

Conclusion: The results demonstrate a promising path for developing general strategic intelligence in complex domains through specialized training approaches.

Abstract: Game playing has long served as a fundamental benchmark for evaluating Artificial General Intelligence. While Large Language Models (LLMs) have demonstrated impressive capabilities in general reasoning, their effectiveness in spatial strategic reasoning, which is critical for complex and fully observable board games, remains insufficiently explored. In this work, we adopt Chinese Chess (Xiangqi) as a challenging and rich testbed due to its intricate rules and spatial complexity. To advance LLMs’ strategic competence in such environments, we propose a training framework tailored to Xiangqi, built upon a large-scale dataset of five million board-move pairs enhanced with expert annotations and engine evaluations. Building on this foundation, we introduce Xiangqi-R1, a 7B-parameter model trained in multi-stage manner. Our Experimental results indicate that, despite their size and power, general-purpose LLMs struggle to achieve satisfactory performance in these tasks. Compared to general-purpose LLMs, Xiangqi-R1 greatly advances with an 18% rise in move legality and a 22% boost in analysis accuracy. Our results point to a promising path for creating general strategic intelligence in complex areas.

Method: Created MoHoBench - a large-scale benchmark with 12k+ visual question samples using multi-stage filtering and human verification, then benchmarked 28 popular MLLMs and implemented alignment methods using supervised and preference learning.

Result: Most models fail to appropriately refuse to answer unanswerable questions, and MLLMs’ honesty is deeply influenced by visual information rather than being solely a language modeling issue.

Conclusion: Honesty in MLLMs requires dedicated multimodal alignment methods, and the paper provides initial alignment approaches and a foundation for future work on trustworthy MLLMs.

Abstract: Recently Multimodal Large Language Models (MLLMs) have achieved considerable advancements in vision-language tasks, yet produce potentially harmful or untrustworthy content. Despite substantial work investigating the trustworthiness of language models, MMLMs’ capability to act honestly, especially when faced with visually unanswerable questions, remains largely underexplored. This work presents the first systematic assessment of honesty behaviors across various MLLMs. We ground honesty in models’ response behaviors to unanswerable visual questions, define four representative types of such questions, and construct MoHoBench, a large-scale MMLM honest benchmark, consisting of 12k+ visual question samples, whose quality is guaranteed by multi-stage filtering and human verification. Using MoHoBench, we benchmarked the honesty of 28 popular MMLMs and conducted a comprehensive analysis. Our findings show that: (1) most models fail to appropriately refuse to answer when necessary, and (2) MMLMs’ honesty is not solely a language modeling issue, but is deeply influenced by visual information, necessitating the development of dedicated methods for multimodal honesty alignment. Therefore, we implemented initial alignment methods using supervised and preference learning to improve honesty behavior, providing a foundation for future work on trustworthy MLLMs. Our data and code can be found at https://github.com/yanxuzhu/MoHoBench.

Method: Introduces an anthropomorphic evaluation paradigm with three-dimensional taxonomy: IQ (General Intelligence), EQ (Alignment Ability), and PQ (Professional Expertise), plus a Value-oriented Evaluation framework assessing economic, social, ethical, and environmental factors.

Result: Analysis of 200+ benchmarks reveals key challenges including dynamic assessment needs and interpretability gaps. Provides actionable guidance for developing technically proficient, contextually relevant, and ethically sound LLMs.

Conclusion: The proposed evaluation framework enables comprehensive assessment of LLMs for real-world deployment, addressing the current disconnect between benchmark metrics and practical utility.

Abstract: For Large Language Models (LLMs), a disconnect persists between benchmark performance and real-world utility. Current evaluation frameworks remain fragmented, prioritizing technical metrics while neglecting holistic assessment for deployment. This survey introduces an anthropomorphic evaluation paradigm through the lens of human intelligence, proposing a novel three-dimensional taxonomy: Intelligence Quotient (IQ)-General Intelligence for foundational capacity, Emotional Quotient (EQ)-Alignment Ability for value-based interactions, and Professional Quotient (PQ)-Professional Expertise for specialized proficiency. For practical value, we pioneer a Value-oriented Evaluation (VQ) framework assessing economic viability, social impact, ethical alignment, and environmental sustainability. Our modular architecture integrates six components with an implementation roadmap. Through analysis of 200+ benchmarks, we identify key challenges including dynamic assessment needs and interpretability gaps. It provides actionable guidance for developing LLMs that are technically proficient, contextually relevant, and ethically sound. We maintain a curated repository of open-source evaluation resources at: https://github.com/onejune2018/Awesome-LLM-Eval.

Method: The authors analyze the complete workflow of agents, propose a new taxonomy of hallucination types occurring at different stages, examine 18 triggering causes, and review existing studies on mitigation and detection approaches.

Result: The survey provides a comprehensive framework for understanding agent hallucinations, categorizing them by workflow stage, identifying root causes, and summarizing current solutions.

Conclusion: This work aims to inspire further research on addressing hallucinations in LLM-based agents to develop more robust and reliable agent systems.

Abstract: Driven by the rapid advancements of Large Language Models (LLMs), LLM-based agents have emerged as powerful intelligent systems capable of human-like cognition, reasoning, and interaction. These agents are increasingly being deployed across diverse real-world applications, including student education, scientific research, and financial analysis. However, despite their remarkable potential, LLM-based agents remain vulnerable to hallucination issues, which can result in erroneous task execution and undermine the reliability of the overall system design. Addressing this critical challenge requires a deep understanding and a systematic consolidation of recent advances on LLM-based agents. To this end, we present the first comprehensive survey of hallucinations in LLM-based agents. By carefully analyzing the complete workflow of agents, we propose a new taxonomy that identifies different types of agent hallucinations occurring at different stages. Furthermore, we conduct an in-depth examination of eighteen triggering causes underlying the emergence of agent hallucinations. Through a detailed review of a large number of existing studies, we summarize approaches for hallucination mitigation and detection, and highlight promising directions for future research. We hope this survey will inspire further efforts toward addressing hallucinations in LLM-based agents, ultimately contributing to the development of more robust and reliable agent systems.

Method: PIXEL learns property-aligned subspaces from dual views (tail-averaged and end-token), selects intervention strength via constrained geometric optimization with closed-form solution, performs orthogonal residual calibration, and uses position-scanning to identify receptive injection sites.

Result: PIXEL consistently improves attribute alignment across diverse models and evaluation paradigms while preserving model general capabilities, demonstrating practical and principled LLM behavior control.

Conclusion: PIXEL offers a practical and principled method for controllable generation in LLMs, providing representation-level guarantees for reliable alignment through minimal-intervention rules.

Abstract: Reliable behavior control is central to deploying large language models (LLMs) on the web. Activation steering offers a tuning-free route to align attributes (e.g., truthfulness) that ensure trustworthy generation. Prevailing approaches rely on coarse heuristics and lack a principled account of where to steer and how strongly to intervene. To this end, we propose Position-wise Injection with eXact Estimated Levels (PIXEL), a position-wise activation steering framework that, in contrast to prior work, learns a property-aligned subspace from dual views (tail-averaged and end-token) and selects intervention strength via a constrained geometric objective with a closed-form solution, thereby adapting to token-level sensitivity without global hyperparameter tuning. PIXEL further performs sample-level orthogonal residual calibration to refine the global attribute direction and employs a lightweight position-scanning routine to identify receptive injection sites. We additionally provide representation-level guarantees for the minimal-intervention rule, supporting reliable alignment. Across diverse models and evaluation paradigms, PIXEL consistently improves attribute alignment while preserving model general capabilities, offering a practical and principled method for LLMs’ controllable generation. Our code is available at https://github.com/V1centNevwake/PIXEL-Adaptive-Steering

Method: Uses multiple Evaluation Agents with LLMs to independently judge PHI extraction correctness, then consolidates results through LLM-based majority voting for stable ranking.

Result: Experiments show TEAM-PHI produces consistent rankings that match supervised evaluation and human judgment, despite individual evaluator variation.

Conclusion: TEAM-PHI provides a practical, secure, and cost-effective solution for automatic PHI de-identification evaluation and model selection when ground-truth labels are limited.

Abstract: Protected health information (PHI) de-identification is critical for enabling the safe reuse of clinical notes, yet evaluating and comparing PHI de-identification models typically depends on costly, small-scale expert annotations. We present TEAM-PHI, a multi-agent evaluation and selection framework that uses large language models (LLMs) to automatically measure de-identification quality and select the best-performing model without heavy reliance on gold labels. TEAM-PHI deploys multiple Evaluation Agents, each independently judging the correctness of PHI extractions and outputting structured metrics. Their results are then consolidated through an LLM-based majority voting mechanism that integrates diverse evaluator perspectives into a single, stable, and reproducible ranking. Experiments on a real-world clinical note corpus demonstrate that TEAM-PHI produces consistent and accurate rankings: despite variation across individual evaluators, LLM-based voting reliably converges on the same top-performing systems. Further comparison with ground-truth annotations and human evaluation confirms that the framework’s automated rankings closely match supervised evaluation. By combining independent evaluation agents with LLM majority voting, TEAM-PHI offers a practical, secure, and cost-effective solution for automatic evaluation and best-model selection in PHI de-identification, even when ground-truth labels are limited.

Method: Multi-stage curriculum reinforcement learning with easy-to-hard curriculum under outcome-only supervision, starting with fine-tuning on high-quality trajectories from outcome-based rejection sampling.

Result: Consistently outperforms prior approaches across three datasets, achieving up to 11.1% relative improvement on GrailQA zero-shot subset while using only 1/12 of training data.

Conclusion: Outcome-only supervision with curriculum RL enables powerful autonomous reasoning behaviors and strong agentic capabilities in KBQA.

Abstract: Knowledge Base Question Answering (KBQA) aims to answer natural-language questions over a structured Knowledge Base (KB). Recent work improves KBQA by adopting an agentic reasoning paradigm, in which Large Language Models (LLMs) iteratively decompose a question, generate its corresponding logical queries, and interact with the KB to derive the answer. However, these methods typically fine-tune LLMs on reasoning trajectories synthesized via process supervision, which offers weak incentives for exploration and thus fails to strengthen the agentic reasoning ability. In this paper, we propose KnowCoder-A1, an LLM that can autonomously perform agentic reasoning on KBs to obtain answers. To incentivize autonomous exploration, KnowCoder-A1 trains the LLM under outcome-only supervision via a multi-stage curriculum reinforcement learning with an easy-to-hard curriculum. To establish foundational agentic capabilities, KnowCoder-A1 first fine-tunes the LLM on a small set of high-quality trajectories obtained through outcome-based rejection sampling. Then, to alleviate the reward sparsity inherent in outcome-only supervision, it applies multi-stage curriculum RL with reward schedules that progress from easy to hard. Trained with outcome-only supervision, KnowCoder-A1 exhibits powerful reasoning behaviors and consistently outperforms prior approaches across three mainstream datasets. Notably, on the zero-shot subset of GrailQA, KnowCoder-A1 achieves up to an 11.1% relative improvement while using only one-twelfth of the training data, demonstrating strong agentic reasoning capabilities.

Method: Transformer-based framework integrating demographic, clinical, and lab data with quantization-based tokenization for continuous values, pretrained with masked language modeling and fine-tuned for binary classification of CKD progression from stage 3a to 5.

Result: Outperformed CEHR-BERT with ROC-AUC up to 0.995 and PR-AUC up to 0.989 for short-term prediction on 91,816 patient cohort.

Conclusion: Transformer architectures with temporal design choices are effective for clinical prognosis modeling and offer promising direction for personalized CKD care.

Abstract: Chronic Kidney Disease (CKD) affects nearly 10% of the global population and often progresses to end-stage renal failure. Accurate prognosis prediction is vital for timely interventions and resource optimization. We present a transformer-based framework for predicting CKD progression using multi-modal electronic health records (EHR) from the Seoul National University Hospital OMOP Common Data Model. Our approach (\textbf{ProQ-BERT}) integrates demographic, clinical, and laboratory data, employing quantization-based tokenization for continuous lab values and attention mechanisms for interpretability. The model was pretrained with masked language modeling and fine-tuned for binary classification tasks predicting progression from stage 3a to stage 5 across varying follow-up and assessment periods. Evaluated on a cohort of 91,816 patients, our model consistently outperformed CEHR-BERT, achieving ROC-AUC up to 0.995 and PR-AUC up to 0.989 for short-term prediction. These results highlight the effectiveness of transformer architectures and temporal design choices in clinical prognosis modeling, offering a promising direction for personalized CKD care.

Method: Developed a principled framework based on shared responsibility between user and system, distinguishing query types and analyzing queries in 15 popular datasets for tabular question answering and analysis.

Result: Found that current datasets mix query types in uncontrolled ways, making them inadequate for evaluating both execution accuracy and interpretation capabilities of systems.

Conclusion: The cooperation framework informs better design and evaluation of natural language interfaces for tabular data, providing concrete directions for future research and broader implications.

Abstract: Natural language interfaces to tabular data must handle ambiguities inherent to queries. Instead of treating ambiguity as a deficiency, we reframe it as a feature of cooperative interaction where users are intentional about the degree to which they specify queries. We develop a principled framework based on a shared responsibility of query specification between user and system, distinguishing unambiguous and ambiguous cooperative queries, which systems can resolve through reasonable inference, from uncooperative queries that cannot be resolved. Applying the framework to evaluations for tabular question answering and analysis, we analyze the queries in 15 popular datasets, and observe an uncontrolled mixing of query types neither adequate for evaluating a system’s execution accuracy nor for evaluating interpretation capabilities. This conceptualization around cooperation in resolving queries informs how to design and evaluate natural language interfaces for tabular data analysis, for which we distill concrete directions for future research and broader implications.

Method: Fine-tuned LLaMA-3.1-8B-Instruct to generate CQs that validate semantic discrepancies by simulating missing/misused axioms through LLM-generated natural language definitions with intentional misalignments (removing axioms or altering logical operators).

Result: The fine-tuned model achieved 26% higher precision and 28.2% higher recall than GPT-4.1 in generating CQs for pitfall validation, and the generated CQs can detect a broader range of modeling errors compared to existing datasets.

Conclusion: This research enables automatic generation of TBox-validating CQs using LLMs, significantly reducing manual effort while improving semantic alignment between ontologies and expert knowledge, representing the first study to target semantic pitfall validation in CQ generation using LLMs.

Abstract: Competency Questions (CQs) play a crucial role in validating ontology design. While manually crafting CQs can be highly time-consuming and costly for ontology engineers, recent studies have explored the use of large language models (LLMs) to automate this process. However, prior approaches have largely evaluated generated CQs based on their similarity to existing datasets, which often fail to verify semantic pitfalls such as “Misusing allValuesFrom”. Since such pitfalls cannot be reliably detected through rule-based methods, we propose a novel dataset and model of Validating Semantic Pitfalls in Ontology (VSPO) for CQ generation specifically designed to verify the semantic pitfalls. To simulate missing and misused axioms, we use LLMs to generate natural language definitions of classes and properties and introduce misalignments between the definitions and the ontology by removing axioms or altering logical operators (e.g., substituting union with intersection). We then fine-tune LLaMA-3.1-8B-Instruct to generate CQs that validate these semantic discrepancies between the provided definitions and the corresponding axioms. The resulting CQs can detect a broader range of modeling errors compared to existing public datasets. Our fine-tuned model demonstrates superior performance over baselines, showing 26% higher precision and 28.2% higher recall than GPT-4.1 in generating CQs for pitfall validation. This research enables automatic generation of TBox-validating CQs using LLMs, significantly reducing manual effort while improving semantic alignment between ontologies and expert knowledge. To the best of our knowledge, this is the first study to target semantic pitfall validation in CQ generation using LLMs.

Method: Extended MLE-Bench with code-sabotage tasks (backdoors, generalization failures), studied sandbagging capabilities, and used LM monitors to detect suspicious behavior while measuring agents’ ability to evade detection.

Result: Frontier agents successfully performed sabotage tasks and could calibrate performance to target levels. Monitors detected code-sabotage well but struggled with sandbagging detection. Multiple monitor aggregation helped but may not be reliable enough for high-stakes domains.

Conclusion: AI agents demonstrate concerning capabilities to sabotage ML systems and manipulate performance while evading detection, highlighting significant safety risks in autonomous ML R&D deployment.

Abstract: AI systems are increasingly able to autonomously conduct realistic software engineering tasks, and may soon be deployed to automate machine learning (ML) R&D itself. Frontier AI systems may be deployed in safety-critical settings, including to help ensure the safety of future systems. Unfortunately, frontier and future systems may not be sufficiently trustworthy, and there is evidence that these systems may even be misaligned with their developers or users. Therefore, we investigate the capabilities of AI agents to act against the interests of their users when conducting ML engineering, by sabotaging ML models, sandbagging their performance, and subverting oversight mechanisms. First, we extend MLE-Bench, a benchmark for realistic ML tasks, with code-sabotage tasks such as implanting backdoors and purposefully causing generalisation failures. Frontier agents make meaningful progress on our sabotage tasks. In addition, we study agent capabilities to sandbag on MLE-Bench. Agents can calibrate their performance to specified target levels below their actual capability. To mitigate sabotage, we use LM monitors to detect suspicious agent behaviour, and we measure model capability to sabotage and sandbag without being detected by these monitors. Overall, monitors are capable at detecting code-sabotage attempts but our results suggest that detecting sandbagging is more difficult. Additionally, aggregating multiple monitor predictions works well, but monitoring may not be sufficiently reliable to mitigate sabotage in high-stakes domains. Our benchmark is implemented in the UK AISI’s Inspect framework and we make our code publicly available at https://github.com/TeunvdWeij/ctrl-alt-deceit

Method: Characterizes requirements for agent decision making through analysis and empirical case studies, examining how agents integrate normative, pragmatic, and situational understanding.

Result: Identifies the types of knowledge agents require to make decisions robust to agent goals and aligned with human expectations in complex, real-world environments.

Conclusion: Agents need to integrate multiple types of contextual knowledge to select and pursue more aligned courses of action when facing novel or under-specified contexts where standard policies are insufficient.

Abstract: Deployed, autonomous AI systems must often evaluate multiple plausible courses of action (extended sequences of behavior) in novel or under-specified contexts. Despite extensive training, these systems will inevitably encounter scenarios where no available course of action fully satisfies all operational constraints (e.g., operating procedures, rules, laws, norms, and goals). To achieve goals in accordance with human expectations and values, agents must go beyond their trained policies and instead construct, evaluate, and justify candidate courses of action. These processes require contextual “knowledge” that may lie outside prior (policy) training. This paper characterizes requirements for agent decision making in these contexts. It also identifies the types of knowledge agents require to make decisions robust to agent goals and aligned with human expectations. Drawing on both analysis and empirical case studies, we examine how agents need to integrate normative, pragmatic, and situational understanding to select and then to pursue more aligned courses of action in complex, real-world environments.

Method: Proposes M-GRPO, a hierarchical extension of Group Relative Policy Optimization for vertical multi-agent systems with a main planner and multiple sub-agents. Uses group-relative advantages for hierarchical credit assignment, trajectory-alignment for fixed-size batches despite variable sub-agent invocations, and decoupled training pipeline with separate servers exchanging minimal statistics.

Result: Outperforms single-agent GRPO and multi-agent GRPO with frozen sub-agents on real-world benchmarks (GAIA, XBench-DeepSearch, WebWalkerQA), demonstrating improved stability and sample efficiency.

Conclusion: Aligning heterogeneous trajectories and decoupling optimization across specialized agents enhances tool-augmented reasoning tasks in multi-agent systems.

Abstract: Multi-agent systems perform well on general reasoning tasks. However, the lack of training in specialized areas hinders their accuracy. Current training methods train a unified large language model (LLM) for all agents in the system. This may limit the performances due to different distributions underlying for different agents. Therefore, training multi-agent systems with distinct LLMs should be the next step to solve. However, this approach introduces optimization challenges. For example, agents operate at different frequencies, rollouts involve varying sub-agent invocations, and agents are often deployed across separate servers, disrupting end-to-end gradient flow. To address these issues, we propose M-GRPO, a hierarchical extension of Group Relative Policy Optimization designed for vertical Multi-agent systems with a main agent (planner) and multiple sub-agents (multi-turn tool executors). M-GRPO computes group-relative advantages for both main and sub-agents, maintaining hierarchical credit assignment. It also introduces a trajectory-alignment scheme that generates fixed-size batches despite variable sub-agent invocations. We deploy a decoupled training pipeline in which agents run on separate servers and exchange minimal statistics via a shared store. This enables scalable training without cross-server backpropagation. In experiments on real-world benchmarks (e.g., GAIA, XBench-DeepSearch, and WebWalkerQA), M-GRPO consistently outperforms both single-agent GRPO and multi-agent GRPO with frozen sub-agents, demonstrating improved stability and sample efficiency. These results show that aligning heterogeneous trajectories and decoupling optimization across specialized agents enhances tool-augmented reasoning tasks.

Method: Three key innovations: speaker identification using audio-visual synchronization, knowledge distillation to transfer textual emotion understanding to audio/visual modalities, and hierarchical attention fusion with composite loss functions.

Result: Achieved superior performance with 67.75% weighted F1 score on MELD and 72.44% on IEMOCAP datasets, with notable improvements on minority emotion classes.

Conclusion: The proposed framework effectively addresses speaker ambiguity and class imbalance in multi-speaker emotion recognition, demonstrating significant performance gains especially for minority emotion categories.

Abstract: Emotion recognition in multi-speaker conversations faces significant challenges due to speaker ambiguity and severe class imbalance. We propose a novel framework that addresses these issues through three key innovations: (1) a speaker identification module that leverages audio-visual synchronization to accurately identify the active speaker, (2) a knowledge distillation strategy that transfers superior textual emotion understanding to audio and visual modalities, and (3) hierarchical attention fusion with composite loss functions to handle class imbalance. Comprehensive evaluations on MELD and IEMOCAP datasets demonstrate superior performance, achieving 67.75% and 72.44% weighted F1 scores respectively, with particularly notable improvements on minority emotion classes.

Method: PRISMA-guided systematic review of ~500 papers, analyzing temporal trends, task types, methods (rankSVM vs RLHF), evaluation strategies, and metric-human judgment alignment.

Result: Found only 30 papers (6%) using preference learning for audio; identified shift from emotion recognition to generation tasks post-2021; revealed misalignment between traditional metrics and human judgments; discovered multi-stage training pipelines combining reward signals.

Conclusion: Preference learning shows promise for capturing subjective audio qualities but requires standardized benchmarks, better datasets, and investigation of temporal factors unique to audio.

Abstract: Despite the parallel challenges that audio and text domains face in evaluating generative model outputs, preference learning remains remarkably underexplored in audio applications. Through a PRISMA-guided systematic review of approximately 500 papers, we find that only 30 (6%) apply preference learning to audio tasks. Our analysis reveals a field in transition: pre-2021 works focused on emotion recognition using traditional ranking methods (rankSVM), while post-2021 studies have pivoted toward generation tasks employing modern RLHF frameworks. We identify three critical patterns: (1) the emergence of multi-dimensional evaluation strategies combining synthetic, automated, and human preferences; (2) inconsistent alignment between traditional metrics (WER, PESQ) and human judgments across different contexts; and (3) convergence on multi-stage training pipelines that combine reward signals. Our findings suggest that while preference learning shows promise for audio, particularly in capturing subjective qualities like naturalness and musicality, the field requires standardized benchmarks, higher-quality datasets, and systematic investigation of how temporal factors unique to audio impact preference learning frameworks.

Method: Uses note event histograms as supervision and an Expectation-Maximization approach to iteratively refine predictions based solely on note occurrence counts, without explicit local alignment.

Result: Experiments on piano, guitar, and multi-instrument datasets show CountEM matches or surpasses existing weakly supervised methods while being more robust, scalable, and efficient.

Conclusion: CountEM significantly reduces annotation efforts while maintaining high transcription accuracy, improving AMT’s practicality for various musical contexts.

Abstract: Automatic Music Transcription (AMT) converts audio recordings into symbolic musical representations. Training deep neural networks (DNNs) for AMT typically requires strongly aligned training pairs with precise frame-level annotations. Since creating such datasets is costly and impractical for many musical contexts, weakly aligned approaches using segment-level annotations have gained traction. However, existing methods often rely on Dynamic Time Warping (DTW) or soft alignment loss functions, both of which still require local semantic correspondences, making them error-prone and computationally expensive. In this article, we introduce CountEM, a novel AMT framework that eliminates the need for explicit local alignment by leveraging note event histograms as supervision, enabling lighter computations and greater flexibility. Using an Expectation-Maximization (EM) approach, CountEM iteratively refines predictions based solely on note occurrence counts, significantly reducing annotation efforts while maintaining high transcription accuracy. Experiments on piano, guitar, and multi-instrument datasets demonstrate that CountEM matches or surpasses existing weakly supervised methods, improving AMT’s robustness, scalability, and efficiency. Our project page is available at https://yoni-yaffe.github.io/count-the-notes.

Method: Proposed lightweight token selection strategy that considers the time dimension to prune tokens while preserving performance.

Result: Retaining only 25% of initial tokens resulted in maximum 2% CIDEr decrease on Clotho v2 and 4% accuracy decrease on MMAU.

Conclusion: Token pruning strategies are effective for audio-language models, significantly reducing computational requirements with minimal performance degradation.

Abstract: Recent audio-language models have shown impressive performance across a wide range of audio tasks and are increasingly capable of handling long audio inputs. However, the computing costs in these models heavily depend on sequence length, which can become very large given the nature of audio data. In the vision-language domain, token pruning methods have proven effective in reducing token counts while preserving strong performance on standard benchmarks. In this work, we investigate the relevance and effectiveness of such token selection strategies in the context of audio-language models. We also improve them by proposing a lightweight strategy that takes the time dimension into account. While retaining only a quarter of the initial tokens, our approach results in a relative maximum decrease of 2% in CIDEr on Clotho v2 and a relative maximum decrease of 4% in accuracy on MMAU.

Method: Leverages recent advances in rectified flow and diffusion transformers to create a zero-shot text-to-music editing model capable of handling diverse editing tasks on real-world music.

Result: Experimental results show MusRec outperforms existing methods in preserving musical content, maintaining structural consistency, and achieving high editing fidelity.

Conclusion: MusRec establishes a strong foundation for controllable music editing in real-world scenarios, addressing limitations of previous approaches.

Abstract: Music editing has emerged as an important and practical area of artificial intelligence, with applications ranging from video game and film music production to personalizing existing tracks according to user preferences. However, existing models face significant limitations, such as being restricted to editing synthesized music generated by their own models, requiring highly precise prompts, or necessitating task-specific retraining, thus lacking true zero-shot capability. leveraging recent advances in rectified flow and diffusion transformers, we introduce MusRec, a zero-shot text-to-music editing model capable of performing diverse editing tasks on real-world music efficiently and effectively. Experimental results demonstrate that our approach outperforms existing methods in preserving musical content, structural consistency, and editing fidelity, establishing a strong foundation for controllable music editing in real-world scenarios.

Method: Uses BEATs SSL backbone for frame-level audio features, classification head for segment-level acoustic event predictions, calibration for event-level predictions, and structured prompts with Qwen2.5-7B-Instruct fine-tuned using GRPO algorithm.

Result: Achieves 62.6% accuracy on the DCASE 2025 Challenge development set for Audio Question Answering.

Conclusion: The combination of acoustic event reasoning with instruction-tuned LLMs is effective for Audio Question Answering tasks.

Abstract: In this report, we describe our submission to Track 5 of the DCASE 2025 Challenge for the task of Audio Question Answering(AQA). Our system leverages the SSL backbone BEATs to extract frame-level audio features, which are then processed by a classification head to generate segment-level predictions of acoustic events, following the Audioset ontology. These segment-level predictions are subsequently calibrated before producing event-level predictions. Finally, these predictions are incorporated into a structured prompt, along with the question and candidate answers. This prompt is then fed to a fine-tuned version of Qwen2.5-7B-Instruct, trained using the GRPO algorithm with a simple reward function. Our method achieves an accuracy of 62.6 % on the development set, demonstrating the effectiveness of combining acoustic event reasoning with instruction-tuned large language models for AQA.

Method: Two core innovations: 1) Amplitude-Aware Linear Attention (MALA) replaces MET module, preserving query norm information to enable efficient global modeling without auxiliary branches; 2) Inception Depthwise Convolution (IDConv) replaces DE module, using parallel branches (square, horizontal, vertical strips) to capture spectrogram features with minimal parameters.

Result: IMSE reduces parameters by 16.8% (from 0.513M to 0.427M) while achieving competitive PESQ score of 3.373 on VoiceBank+DEMAND dataset, comparable to state-of-the-art performance.

Conclusion: The study establishes a new benchmark for balancing model size and speech quality in ultra-lightweight speech enhancement, demonstrating that systematic optimization can achieve significant parameter reduction without sacrificing performance.

Abstract: Achieving a balance between lightweight design and high performance remains a significant challenge for speech enhancement (SE) tasks on resource-constrained devices. Existing state-of-the-art methods, such as MUSE, have established a strong baseline with only 0.51M parameters by introducing a Multi-path Enhanced Taylor (MET) transformer and Deformable Embedding (DE). However, an in-depth analysis reveals that MUSE still suffers from efficiency bottlenecks: the MET module relies on a complex “approximate-compensate” mechanism to mitigate the limitations of Taylor-expansion-based attention, while the offset calculation for deformable embedding introduces additional computational burden. This paper proposes IMSE, a systematically optimized and ultra-lightweight network. We introduce two core innovations: 1) Replacing the MET module with Amplitude-Aware Linear Attention (MALA). MALA fundamentally rectifies the “amplitude-ignoring” problem in linear attention by explicitly preserving the norm information of query vectors in the attention calculation, achieving efficient global modeling without an auxiliary compensation branch. 2) Replacing the DE module with Inception Depthwise Convolution (IDConv). IDConv borrows the Inception concept, decomposing large-kernel operations into efficient parallel branches (square, horizontal, and vertical strips), thereby capturing spectrogram features with extremely low parameter redundancy. Extensive experiments on the VoiceBank+DEMAND dataset demonstrate that, compared to the MUSE baseline, IMSE significantly reduces the parameter count by 16.8% (from 0.513M to 0.427M) while achieving competitive performance comparable to the state-of-the-art on the PESQ metric (3.373). This study sets a new benchmark for the trade-off between model size and speech quality in ultra-lightweight speech enhancement.

Method: Proposed CPFG-Net neural network with transformation algorithm mapping perceptual features to chord representations. Trained on BCPT-220K dataset from classical music for controllable prediction of perceptual features and tonal structures.

Result: State-of-the-art perceptual feature prediction capability. Demonstrates musical expressiveness and creativity in chord inference with harmonically coherent chord progressions.

Conclusion: Offers novel perspective on melody harmonization, contributes to broader music generation tasks, and can be extended to audio-based models.

Abstract: While Large Language Models (LLMs) make symbolic music generation increasingly accessible, producing music with distinctive composition and rich expressiveness remains a significant challenge. Many studies have introduced emotion models to guide the generative process. However, these approaches still fall short of delivering novelty and creativity. In the field of Music Information Retrieval (MIR), auditory perception is recognized as a key dimension of musical experience, offering insights into both compositional intent and emotional patterns. To this end, we propose a neural network named CPFG-Net, along with a transformation algorithm that maps perceptual feature values to chord representations, enabling melody harmonization. The system can controllably predict sequences of perceptual features and tonal structures from given melodies, and subsequently generate harmonically coherent chord progressions. Our network is trained on our newly constructed perceptual feature dataset BCPT-220K, derived from classical music. Experimental results show state-of-the-art perceptual feature prediction capability of our model as well as demonstrate our musical expressiveness and creativity in chord inference. This work offers a novel perspective on melody harmonization and contributes to broader music generation tasks. Our symbolic-based model can be easily extended to audio-based models.

Method: HIN framework applies acoustic modifications to raw audio waveforms, including temporal dynamics alterations and strategic injection of spectrally tailored noise, creating consistent patterns captured by ALLM’s acoustic feature encoder.

Result: Experiments show: (I) audio features achieve over 90% attack success rate, (II) ALLMs show significant sensitivity differences across acoustic features with minimal response to volume triggers, and (III) poisoned samples cause only marginal loss curve fluctuations.

Conclusion: ALLMs exhibit critical vulnerabilities to audio-feature-based backdoor attacks, with HIN demonstrating high effectiveness and stealth in compromising model safety.

Abstract: As Audio Large Language Models (ALLMs) emerge as powerful tools for speech processing, their safety implications demand urgent attention. While considerable research has explored textual and vision safety, audio’s distinct characteristics present significant challenges. This paper first investigates: Is ALLM vulnerable to backdoor attacks exploiting acoustic triggers? In response to this issue, we introduce Hidden in the Noise (HIN), a novel backdoor attack framework designed to exploit subtle, audio-specific features. HIN applies acoustic modifications to raw audio waveforms, such as alterations to temporal dynamics and strategic injection of spectrally tailored noise. These changes introduce consistent patterns that an ALLM’s acoustic feature encoder captures, embedding robust triggers within the audio stream. To evaluate ALLM robustness against audio-feature-based triggers, we develop the AudioSafe benchmark, assessing nine distinct risk types. Extensive experiments on AudioSafe and three established safety datasets reveal critical vulnerabilities in existing ALLMs: (I) audio features like environment noise and speech rate variations achieve over 90% average attack success rate. (II) ALLMs exhibit significant sensitivity differences across acoustic features, particularly showing minimal response to volume as a trigger, and (III) poisoned sample inclusion causes only marginal loss curve fluctuations, highlighting the attack’s stealth.

Method: Conducted probing analysis on AudioLDM 2 attention maps, revealing cross-attention handles musical characteristics while self-attention preserves temporal structure. Melodia selectively manipulates self-attention maps during denoising and uses an attention repository to store source music information.

Result: Both objective and subjective experiments show superior performance in textual adherence and structural integrity across various datasets compared to existing methods.

Conclusion: The research enhances understanding of music generation model mechanisms and provides improved control for music creation through selective attention manipulation.

Abstract: Text-to-music generation technology is progressing rapidly, creating new opportunities for musical composition and editing. However, existing music editing methods often fail to preserve the source music’s temporal structure, including melody and rhythm, when altering particular attributes like instrument, genre, and mood. To address this challenge, this paper conducts an in-depth probing analysis on attention maps within AudioLDM 2, a diffusion-based model commonly used as the backbone for existing music editing methods. We reveal a key finding: cross-attention maps encompass details regarding distinct musical characteristics, and interventions on these maps frequently result in ineffective modifications. In contrast, self-attention maps are essential for preserving the temporal structure of the source music during its conversion into the target music. Building upon this understanding, we present Melodia, a training-free technique that selectively manipulates self-attention maps in particular layers during the denoising process and leverages an attention repository to store source music information, achieving accurate modification of musical characteristics while preserving the original structure without requiring textual descriptions of the source music. Additionally, we propose two novel metrics to better evaluate music editing methods. Both objective and subjective experiments demonstrate that our approach achieves superior results in terms of textual adherence and structural integrity across various datasets. This research enhances comprehension of internal mechanisms within music generation models and provides improved control for music creation.

Method: Proposed Spectral Masking and Interpolation Attack (SMIA) that strategically manipulates inaudible frequency regions of AI-generated audio to create adversarial samples that sound authentic but deceive countermeasures.

Result: SMIA achieved 82% success rate against combined VAS/CM systems, 97.5% against standalone speaker verification, and 100% against countermeasures, demonstrating current security postures are insufficient.

Conclusion: Urgent need for paradigm shift toward next-generation dynamic, context-aware defenses that can evolve with the threat landscape.

Abstract: Voice Authentication Systems (VAS) use unique vocal characteristics for verification. They are increasingly integrated into high-security sectors such as banking and healthcare. Despite their improvements using deep learning, they face severe vulnerabilities from sophisticated threats like deepfakes and adversarial attacks. The emergence of realistic voice cloning complicates detection, as systems struggle to distinguish authentic from synthetic audio. While anti-spoofing countermeasures (CMs) exist to mitigate these risks, many rely on static detection models that can be bypassed by novel adversarial methods, leaving a critical security gap. To demonstrate this vulnerability, we propose the Spectral Masking and Interpolation Attack (SMIA), a novel method that strategically manipulates inaudible frequency regions of AI-generated audio. By altering the voice in imperceptible zones to the human ear, SMIA creates adversarial samples that sound authentic while deceiving CMs. We conducted a comprehensive evaluation of our attack against state-of-the-art (SOTA) models across multiple tasks, under simulated real-world conditions. SMIA achieved a strong attack success rate (ASR) of at least 82% against combined VAS/CM systems, at least 97.5% against standalone speaker verification systems, and 100% against countermeasures. These findings conclusively demonstrate that current security postures are insufficient against adaptive adversarial attacks. This work highlights the urgent need for a paradigm shift toward next-generation defenses that employ dynamic, context-aware frameworks capable of evolving with the threat landscape.

Method: Two-stage pipeline: 1) Discriminator model filters out low-quality forgeries that fail to reproduce vocal likeness accurately; 2) Singer identification model trained exclusively on authentic recordings identifies singers in remaining high-quality deepfakes and authentic audio.

Result: The system consistently outperforms existing baselines on both authentic and synthetic content, demonstrating effective singer identification in high-quality vocal deepfakes.

Conclusion: The proposed two-stage approach provides an effective solution for singer identification in vocal deepfakes, particularly targeting the most harmful high-quality forgeries while leveraging authentic training data for reliable performance.

Abstract: The proliferation of highly realistic singing voice deepfakes presents a significant challenge to protecting artist likeness and content authenticity. Automatic singer identification in vocal deepfakes is a promising avenue for artists and rights holders to defend against unauthorized use of their voice, but remains an open research problem. Based on the premise that the most harmful deepfakes are those of the highest quality, we introduce a two-stage pipeline to identify a singer’s vocal likeness. It first employs a discriminator model to filter out low-quality forgeries that fail to accurately reproduce vocal likeness. A subsequent model, trained exclusively on authentic recordings, identifies the singer in the remaining high-quality deepfakes and authentic audio. Experiments show that this system consistently outperforms existing baselines on both authentic and synthetic content.

Method: Extended PINN (XPINN) framework with dynamic domain decomposition into evolving pre-shock and post-shock regions. Uses localized subnetworks for different flow behaviors and enforces flux continuity via Rankine-Hugoniot jump conditions across moving shock interfaces.

Result: XPINN accurately captures discontinuous saturation fronts and compound wave interactions without artificial diffusion or entropy corrections. Achieves superior stability, faster convergence, and better resolution of nonlinear wave dynamics compared to standard PINNs, using smaller models with fewer parameters.

Conclusion: XPINN is an effective and scalable tool for solving challenging hyperbolic PDEs in multiphase flow problems, offering improved performance over standard PINNs for problems with discontinuities and complex wave interactions.

Abstract: The accurate solution of nonlinear hyperbolic partial differential equations (PDEs) remains a central challenge in computational science due to the presence of steep gradients, discontinuities, and multiscale structures that make conventional discretization-based solvers computationally demanding. Physics-Informed Neural Networks (PINNs) embed the governing equations into the learning process, enabling mesh-free solution of PDEs, yet they often struggle to capture steep gradients, discontinuities, and complex nonlinear wave interactions. To address these limitations, this study employs the Extended Physics-Informed Neural Network (XPINN) framework to solve the nonlinear Buckley-Leverett equation with a nonconvex flux function, which models immiscible two-phase flow in porous media. The computational domain is dynamically decomposed in space and time into evolving pre-shock and post-shock regions, allowing localized subnetworks to efficiently learn distinct flow behaviors. Coupling between subnetworks is achieved through the Rankine-Hugoniot jump condition, which enforces physically consistent flux continuity across the moving shock interface. Numerical experiments demonstrate that the proposed XPINN approach accurately captures discontinuous saturation fronts and compound wave interactions without requiring artificial diffusion or entropy corrections. Compared to standard PINNs, the XPINN framework achieves superior stability, faster convergence, and enhanced resolution of nonlinear wave dynamics using smaller, domain-specific models with fewer trainable parameters, establishing it as an effective and scalable tool for solving challenging hyperbolic PDEs in multiphase flow problems. The code of this work is available on github.com/saifkhanengr/XPINN-for-Buckley-Leverett.

Method: Simulated over 6,000 multi-turn attacker-target exchanges across LLM families (0.6B-120B parameters) using JailbreakBench tasks, measuring harm scores and refusal behavior via three independent LLM judges.

Result: Strong correlation between mean harm and log of attacker-to-target size ratio (r=0.51), higher variance in attacker behavior than target susceptibility, and strong negative correlation between attacker refusal frequency and harm.

Conclusion: Size asymmetry influences adversarial robustness, revealing scaling patterns that motivate further investigation into inter-model alignment and safety.

Abstract: Large language models (LLMs) increasingly operate in multi-agent and safety-critical settings, raising open questions about how their vulnerabilities scale when models interact adversarially. This study examines whether larger models can systematically jailbreak smaller ones - eliciting harmful or restricted behavior despite alignment safeguards. Using standardized adversarial tasks from JailbreakBench, we simulate over 6,000 multi-turn attacker-target exchanges across major LLM families and scales (0.6B-120B parameters), measuring both harm score and refusal behavior as indicators of adversarial potency and alignment integrity. Each interaction is evaluated through aggregated harm and refusal scores assigned by three independent LLM judges, providing a consistent, model-based measure of adversarial outcomes. Aggregating results across prompts, we find a strong and statistically significant correlation between mean harm and the logarithm of the attacker-to-target size ratio (Pearson r = 0.51, p < 0.001; Spearman rho = 0.52, p < 0.001), indicating that relative model size correlates with the likelihood and severity of harmful completions. Mean harm score variance is higher across attackers (0.18) than across targets (0.10), suggesting that attacker-side behavioral diversity contributes more to adversarial outcomes than target susceptibility. Attacker refusal frequency is strongly and negatively correlated with harm (rho = -0.93, p < 0.001), showing that attacker-side alignment mitigates harmful responses. These findings reveal that size asymmetry influences robustness and provide exploratory evidence for adversarial scaling patterns, motivating more controlled investigations into inter-model alignment and safety.

Method: BLUE gradually reduces GPS coordinate precision to create hierarchical patches, uses a pyramid encoder-decoder with Transformers at each level, and is trained with trajectory reconstruction using MSE loss.

Result: BLUE consistently outperforms 8 SOTA baselines across 3 downstream tasks, achieving 30.90% higher accuracy on average compared to the best-performing baselines.

Conclusion: The hierarchical patch approach effectively captures both detailed and overall trajectory patterns, demonstrating superior performance in trajectory representation learning.

Abstract: Trajectory representation learning (TRL) maps trajectories to vector embeddings and facilitates tasks such as trajectory classification and similarity search. State-of-the-art (SOTA) TRL methods transform raw GPS trajectories to grid or road trajectories to capture high-level travel semantics, i.e., regions and roads. However, they lose fine-grained spatial-temporal details as multiple GPS points are grouped into a single grid cell or road segment. To tackle this problem, we propose the BLUrred Encoding method, dubbed BLUE, which gradually reduces the precision of GPS coordinates to create hierarchical patches with multiple levels. The low-level patches are small and preserve fine-grained spatial-temporal details, while the high-level patches are large and capture overall travel patterns. To complement different patch levels with each other, our BLUE is an encoder-decoder model with a pyramid structure. At each patch level, a Transformer is used to learn the trajectory embedding at the current level, while pooling prepares inputs for the higher level in the encoder, and up-resolution provides guidance for the lower level in the decoder. BLUE is trained using the trajectory reconstruction task with the MSE loss. We compare BLUE with 8 SOTA TRL methods for 3 downstream tasks, the results show that BLUE consistently achieves higher accuracy than all baselines, outperforming the best-performing baselines by an average of 30.90%. Our code is available at https://github.com/slzhou-xy/BLUE.

Method: Proposes Gradient-Feature Alignment (GFA) regularization applied across multiple layers, which aligns input gradients with internal feature representations to suppress adversarial vulnerability by reducing loss variation in tangential directions.

Result: Significant robustness improvements: 15.2% better than adversarial training under APGD attacks and 24.7% better under FGSM attacks on CIFAR-10. Against optimization-based attacks like DeepFool and EADEN, requires 20-30 times higher perturbation magnitudes to cause misclassification.

Conclusion: DeepDefense is architecture-agnostic, simple to implement, and highly effective, offering a promising direction for improving adversarial robustness of deep learning models through gradient-feature alignment.

Abstract: Deep neural networks are known to be vulnerable to adversarial perturbations, which are small and carefully crafted inputs that lead to incorrect predictions. In this paper, we propose DeepDefense, a novel defense framework that applies Gradient-Feature Alignment (GFA) regularization across multiple layers to suppress adversarial vulnerability. By aligning input gradients with internal feature representations, DeepDefense promotes a smoother loss landscape in tangential directions, thereby reducing the model’s sensitivity to adversarial noise. We provide theoretical insights into how adversarial perturbation can be decomposed into radial and tangential components and demonstrate that alignment suppresses loss variation in tangential directions, where most attacks are effective. Empirically, our method achieves significant improvements in robustness across both gradient-based and optimization-based attacks. For example, on CIFAR-10, CNN models trained with DeepDefense outperform standard adversarial training by up to 15.2% under APGD attacks and 24.7% under FGSM attacks. Against optimization-based attacks such as DeepFool and EADEN, DeepDefense requires 20 to 30 times higher perturbation magnitudes to cause misclassification, indicating stronger decision boundaries and a flatter loss landscape. Our approach is architecture-agnostic, simple to implement, and highly effective, offering a promising direction for improving the adversarial robustness of deep learning models.

Method: SCALEX extracts semantically meaningful directions from H-space using only natural language prompts, enabling zero-shot interpretation without retraining or labeling.

Result: SCALEX detects gender bias in profession prompts, ranks semantic alignment across identity descriptors, and reveals clustered conceptual structure without supervision.

Conclusion: SCALEX makes bias analysis in diffusion models more scalable, interpretable, and extensible by linking prompts to latent directions directly.

Abstract: Image generation models frequently encode social biases, including stereotypes tied to gender, race, and profession. Existing methods for analyzing these biases in diffusion models either focus narrowly on predefined categories or depend on manual interpretation of latent directions. These constraints limit scalability and hinder the discovery of subtle or unanticipated patterns. We introduce SCALEX, a framework for scalable and automated exploration of diffusion model latent spaces. SCALEX extracts semantically meaningful directions from H-space using only natural language prompts, enabling zero-shot interpretation without retraining or labelling. This allows systematic comparison across arbitrary concepts and large-scale discovery of internal model associations. We show that SCALEX detects gender bias in profession prompts, ranks semantic alignment across identity descriptors, and reveals clustered conceptual structure without supervision. By linking prompts to latent directions directly, SCALEX makes bias analysis in diffusion models more scalable, interpretable, and extensible than prior approaches.

Method: Proposes Fair-GNE, which models MARL as a constrained generalized Nash equilibrium-seeking game where agents share resources and individual actions affect others. The framework steers group policy to safe and locally efficient equilibria where no agent can unilaterally improve its utility.

Result: Significant improvement in workload balance over fixed-penalty baselines (0.89 vs. 0.33 JFI, p < 0.01) while maintaining 86% task success rate in high-fidelity resuscitation simulator experiments.

Conclusion: Fair-GNE demonstrates statistically significant fairness gains through adaptive constraint enforcement, providing principled fairness enforcement in large multi-agent learning-based healthcare systems with clear formulations and equilibrium-seeking innovations.

Abstract: Enforcing a fair workload allocation among multiple agents tasked to achieve an objective in learning enabled demand side healthcare worker settings is crucial for consistent and reliable performance at runtime. Existing multi-agent reinforcement learning (MARL) approaches steer fairness by shaping reward through post hoc orchestrations, leaving no certifiable self-enforceable fairness that is immutable by individual agents at runtime. Contextualized within a setting where each agent shares resources with others, we address this shortcoming with a learning enabled optimization scheme among self-interested decision makers whose individual actions affect those of other agents. This extends the problem to a generalized Nash equilibrium (GNE) game-theoretic framework where we steer group policy to a safe and locally efficient equilibrium, so that no agent can improve its utility function by unilaterally changing its decisions. Fair-GNE models MARL as a constrained generalized Nash equilibrium-seeking (GNE) game, prescribing an ideal equitable collective equilibrium within the problem’s natural fabric. Our hypothesis is rigorously evaluated in our custom-designed high-fidelity resuscitation simulator. Across all our numerical experiments, Fair-GNE achieves significant improvement in workload balance over fixed-penalty baselines (0.89 vs.\ 0.33 JFI, $p < 0.01$) while maintaining 86% task success, demonstrating statistically significant fairness gains through adaptive constraint enforcement. Our results communicate our formulations, evaluation metrics, and equilibrium-seeking innovations in large multi-agent learning-based healthcare systems with clarity and principled fairness enforcement.

Method: Region-based channel selection groups EEG channels by functional brain regions, then applies three feature extraction methods (CSP, Fuzzy C-means, TSM) to capture spatial, clustering, and non-linear patterns, with SVM classification.

Result: Achieved classification accuracies of 90.77% on dataset IVA and 84.50% on dataset I from BCI competition III and IV, outperforming existing methods.

Conclusion: The proposed region-specific channel selection and multi-domain feature fusion approach effectively improves motor imagery classification accuracy while reducing computational complexity in BCI systems.

Abstract: A Brain Computer Interface (BCI) connects the human brain to the outside world, providing a direct communication channel. Electroencephalography (EEG) signals are commonly used in BCIs to reflect cognitive patterns related to motor function activities. However, due to the multichannel nature of EEG signals, explicit information processing is crucial to lessen computational complexity in BCI systems. This study proposes an innovative method based on brain region-specific channel selection and multi-domain feature fusion to improve classification accuracy. The novelty of the proposed approach lies in region-based channel selection, where EEG channels are grouped according to their functional relevance to distinct brain regions. By selecting channels based on specific regions involved in motor imagery (MI) tasks, this technique eliminates irrelevant channels, reducing data dimensionality and improving computational efficiency. This also ensures that the extracted features are more reflective of the brain actual activity related to motor tasks. Three distinct feature extraction methods Common Spatial Pattern (CSP), Fuzzy C-means clustering, and Tangent Space Mapping (TSM), are applied to each group of channels based on their brain region. Each method targets different characteristics of the EEG signal: CSP focuses on spatial patterns, Fuzzy C means identifies clusters within the data, and TSM captures non-linear patterns in the signal. The combined feature vector is used to classify motor imagery tasks (left hand, right hand, and right foot) using Support Vector Machine (SVM). The proposed method was validated on publicly available benchmark EEG datasets (IVA and I) from the BCI competition III and IV. The results show that the approach outperforms existing methods, achieving classification accuracies of 90.77% and 84.50% for datasets IVA and I, respectively.

Method: Proposed a one-feature differential evolution attack that perturbs a single kinematic feature of surrounding vehicles within LLM input prompts under black-box settings, tested on the highD dataset.

Result: Experiments show even minor, physically plausible perturbations can significantly disrupt model outputs, revealing high susceptibility to adversarial manipulation and a trade-off between accuracy and robustness.

Conclusion: This study provides the first insights into adversarial vulnerabilities of LLM-driven automated vehicle models and highlights the need for robustness-oriented design in future LLM-based intelligent transportation systems.

Abstract: The integration of large language models (LLMs) into automated driving systems has opened new possibilities for reasoning and decision-making by transforming complex driving contexts into language-understandable representations. Recent studies demonstrate that fine-tuned LLMs can accurately predict vehicle trajectories and lane-change intentions by gathering and transforming data from surrounding vehicles. However, the robustness of such LLM-based prediction models for safety-critical driving systems remains unexplored, despite the increasing concerns about the trustworthiness of LLMs. This study addresses this gap by conducting a systematic vulnerability analysis of LLM-enabled vehicle trajectory prediction. We propose a one-feature differential evolution attack that perturbs a single kinematic feature of surrounding vehicles within the LLM’s input prompts under a black-box setting. Experiments on the highD dataset reveal that even minor, physically plausible perturbations can significantly disrupt model outputs, underscoring the susceptibility of LLM-based predictors to adversarial manipulation. Further analyses reveal a trade-off between accuracy and robustness, examine the failure mechanism, and explore potential mitigation solutions. The findings provide the very first insights into adversarial vulnerabilities of LLM-driven automated vehicle models in the context of vehicular interactions and highlight the need for robustness-oriented design in future LLM-based intelligent transportation systems.

Method: Introduced controllable contrastive learning module to precisely manipulate alignment strength during training and tested on synthetic/real datasets with varying data characteristics.

Result: Impact of explicit alignment depends on data redundancy - optimal alignment strength exists that balances modality-specific signals and shared redundancy in mixed information distributions.

Conclusion: Provides practical guidance on when and how to apply explicit alignment for optimal unimodal encoder performance based on data characteristics.

Abstract: Multimodal learning often relies on aligning representations across modalities to enable effective information integration, an approach traditionally assumed to be universally beneficial. However, prior research has primarily taken an observational approach, examining naturally occurring alignment in multimodal data and exploring its correlation with model performance, without systematically studying the direct effects of explicitly enforced alignment between representations of different modalities. In this work, we investigate how explicit alignment influences both model performance and representation alignment under different modality-specific information structures. Specifically, we introduce a controllable contrastive learning module that enables precise manipulation of alignment strength during training, allowing us to explore when explicit alignment improves or hinders performance. Our results on synthetic and real datasets under different data characteristics show that the impact of explicit alignment on the performance of unimodal models is related to the characteristics of the data: the optimal level of alignment depends on the amount of redundancy between the different modalities. We identify an optimal alignment strength that balances modality-specific signals and shared redundancy in the mixed information distributions. This work provides practical guidance on when and how explicit alignment should be applied to achieve optimal unimodal encoder performance.

Method: Proposes Adaptive Redundancy Regulation (RedReg) with: 1) redundancy phase monitor using effective gain growth rate and redundancy criteria, 2) co-information gating to estimate dominant modality contributions, 3) orthogonal gradient projection and suppression based on redundancy.

Result: Experiments show RedReg outperforms current major methods in most scenarios, with ablation studies confirming the effectiveness of the proposed components.

Conclusion: RedReg successfully addresses modality bias by adaptively regulating redundant information while preserving modality-specific features, demonstrating superior performance across various multimodal learning scenarios.

Abstract: Multimodal learning aims to improve performance by leveraging data from multiple sources. During joint multimodal training, due to modality bias, the advantaged modality often dominates backpropagation, leading to imbalanced optimization. Existing methods still face two problems: First, the long-term dominance of the dominant modality weakens representation-output coupling in the late stages of training, resulting in the accumulation of redundant information. Second, previous methods often directly and uniformly adjust the gradients of the advantaged modality, ignoring the semantics and directionality between modalities. To address these limitations, we propose Adaptive Redundancy Regulation for Balanced Multimodal Information Refinement (RedReg), which is inspired by information bottleneck principle. Specifically, we construct a redundancy phase monitor that uses a joint criterion of effective gain growth rate and redundancy to trigger intervention only when redundancy is high. Furthermore, we design a co-information gating mechanism to estimate the contribution of the current dominant modality based on cross-modal semantics. When the task primarily relies on a single modality, the suppression term is automatically disabled to preserve modality-specific information. Finally, we project the gradient of the dominant modality onto the orthogonal complement of the joint multimodal gradient subspace and suppress the gradient according to redundancy. Experiments show that our method demonstrates superiority among current major methods in most scenarios. Ablation experiments verify the effectiveness of our method. The code is available at https://github.com/xia-zhe/RedReg.git

Method: FLARE integrates: (1) multi-dimensional reputation scoring (performance, statistics, temporal behavior), (2) self-calibrating adaptive thresholds, (3) reputation-weighted aggregation with soft exclusion, and (4) Local Differential Privacy for privatized client updates. Also introduces Statistical Mimicry attack for benchmarking.

Result: Experiments with 100 clients on MNIST, CIFAR-10, and SVHN show FLARE maintains high accuracy, converges faster than state-of-the-art methods, improves robustness by up to 16%, preserves convergence within 30% of non-attacked baseline, and achieves strong detection with minimal overhead.

Conclusion: FLARE provides effective adaptive defense against diverse Byzantine attacks in federated learning through continuous trust evaluation and reputation-based mechanisms, outperforming existing methods while maintaining model performance.

Abstract: Federated learning (FL) enables collaborative model training while preserving data privacy. However, it remains vulnerable to malicious clients who compromise model integrity through Byzantine attacks, data poisoning, or adaptive adversarial behaviors. Existing defense mechanisms rely on static thresholds and binary classification, failing to adapt to evolving client behaviors in real-world deployments. We propose FLARE, an adaptive reputation-based framework that transforms client reliability assessment from binary decisions to a continuous, multi-dimensional trust evaluation. FLARE integrates: (i) a multi-dimensional reputation score capturing performance consistency, statistical anomaly indicators, and temporal behavior, (ii) a self-calibrating adaptive threshold mechanism that adjusts security strictness based on model convergence and recent attack intensity, (iii) reputation-weighted aggregation with soft exclusion to proportionally limit suspicious contributions rather than eliminating clients outright, and (iv) a Local Differential Privacy (LDP) mechanism enabling reputation scoring on privatized client updates. We further introduce a highly evasive Statistical Mimicry (SM) attack, a benchmark adversary that blends honest gradients with synthetic perturbations and persistent drift to remain undetected by traditional filters. Extensive experiments with 100 clients on MNIST, CIFAR-10, and SVHN demonstrate that FLARE maintains high model accuracy and converges faster than state-of-the-art Byzantine-robust methods under diverse attack types, including label flipping, gradient scaling, adaptive attacks, ALIE, and SM. FLARE improves robustness by up to 16% and preserves model convergence within 30% of the non-attacked baseline, while achieving strong malicious-client detection performance with minimal computational overhead. https://github.com/Anonymous0-0paper/FLARE

Method: Adaptation of deep lattice networks (DLN) with LSTM embedding layers for simultaneous quantile regression. The approach spreads quantile inputs to all sub-lattices with extended output size, leveraging DLN’s monotonic constraintability to prevent quantile crossovers.

Result: Experiments on solar irradiance forecasting show the adapted DLN performs as well or better than unconstrained approaches, and outperforms scalable monotonic neural networks.

Conclusion: The adaptation of DLNs successfully enables implicit CDF forecasting without quantile crossovers, and the authors hope to stimulate more research connecting monotonic neural networks with probabilistic forecasting.

Abstract: Probabilistic forecasting is not only a way to add more information to a prediction of the future, but it also builds on weaknesses in point prediction. Sudden changes in a time series can still be captured by a cumulative distribution function (CDF), while a point prediction is likely to miss it entirely. The modeling of CDFs within forecasts has historically been limited to parametric approaches, but due to recent advances, this no longer has to be the case. We aim to advance the fields of probabilistic forecasting and monotonic networks by connecting them and propose an approach that permits the forecasting of implicit, complete, and nonparametric CDFs. For this purpose, we propose an adaptation to deep lattice networks (DLN) for monotonically constrained simultaneous/implicit quantile regression in time series forecasting. Quantile regression usually produces quantile crossovers, which need to be prevented to achieve a legitimate CDF. By leveraging long short term memory units (LSTM) as the embedding layer, and spreading quantile inputs to all sub-lattices of a DLN with an extended output size, we can produce a multi-horizon forecast of an implicit CDF due to the monotonic constraintability of DLNs that prevent quantile crossovers. We compare and evaluate our approach’s performance to relevant state of the art within the context of a highly relevant application of time series forecasting: Day-ahead, hourly forecasts of solar irradiance observations. Our experiments show that the adaptation of a DLN performs just as well or even better than an unconstrained approach. Further comparison of the adapted DLN against a scalable monotonic neural network shows that our approach performs better. With this adaptation of DLNs, we intend to create more interest and crossover investigations in techniques of monotonic neural networks and probabilistic forecasting.

Method: Created VitalBench benchmark with data from 4,000+ surgeries across two medical centers, featuring three evaluation tracks (complete data, incomplete data, cross-center generalization) with masked loss techniques and minimal preprocessing to reflect real-world clinical complexities.

Result: Developed a standardized platform that enables robust and unbiased model evaluation, allowing researchers to focus on architectural innovation while ensuring consistent data handling across different clinical scenarios.

Conclusion: VitalBench provides a foundation for advancing intraoperative vital sign forecasting models that are not only accurate but also robust and adaptable across diverse clinical environments, with publicly available code and data.

Abstract: Intraoperative monitoring and prediction of vital signs are critical for ensuring patient safety and improving surgical outcomes. Despite recent advances in deep learning models for medical time-series forecasting, several challenges persist, including the lack of standardized benchmarks, incomplete data, and limited cross-center validation. To address these challenges, we introduce VitalBench, a novel benchmark specifically designed for intraoperative vital sign prediction. VitalBench includes data from over 4,000 surgeries across two independent medical centers, offering three evaluation tracks: complete data, incomplete data, and cross-center generalization. This framework reflects the real-world complexities of clinical practice, minimizing reliance on extensive preprocessing and incorporating masked loss techniques for robust and unbiased model evaluation. By providing a standardized and unified platform for model development and comparison, VitalBench enables researchers to focus on architectural innovation while ensuring consistency in data handling. This work lays the foundation for advancing predictive models for intraoperative vital sign forecasting, ensuring that these models are not only accurate but also robust and adaptable across diverse clinical environments. Our code and data are available at https://github.com/XiudingCai/VitalBench.

Method: Built on transformer architecture, pre-trained with masking techniques, and fine-tuned on a large-scale dataset of valid/invalid molecular pairs constructed for this purpose.

Result: ChemFixer improved molecular validity across diverse generative models while preserving chemical and biological distributional properties, effectively recovering previously ungeneratable molecules and expanding candidate diversity.

Conclusion: ChemFixer is a practical tool that enhances molecular validity, expands accessible chemical space, and shows extensibility to various downstream tasks including drug-target interaction prediction in data-limited scenarios.

Abstract: Deep learning-based molecular generation models have shown great potential in efficiently exploring vast chemical spaces by generating potential drug candidates with desired properties. However, these models often produce chemically invalid molecules, which limits the usable scope of the learned chemical space and poses significant challenges for practical applications. To address this issue, we propose ChemFixer, a framework designed to correct invalid molecules into valid ones. ChemFixer is built on a transformer architecture, pre-trained using masking techniques, and fine-tuned on a large-scale dataset of valid/invalid molecular pairs that we constructed. Through comprehensive evaluations across diverse generative models, ChemFixer improved molecular validity while effectively preserving the chemical and biological distributional properties of the original outputs. This indicates that ChemFixer can recover molecules that could not be previously generated, thereby expanding the diversity of potential drug candidates. Furthermore, ChemFixer was effectively applied to a drug-target interaction (DTI) prediction task using limited data, improving the validity of generated ligands and discovering promising ligand-protein pairs. These results suggest that ChemFixer is not only effective in data-limited scenarios, but also extensible to a wide range of downstream tasks. Taken together, ChemFixer shows promise as a practical tool for various stages of deep learning-based drug discovery, enhancing molecular validity and expanding accessible chemical space.

Method: Iterative self-training with MA-VLMs that simulate moderator and user perspectives, using PNU loss to handle agreed and disagreed pseudo-labels from unlabeled data.

Result: Substantially outperforms baselines under limited supervision and approaches large-scale model performance on benchmark datasets.

Conclusion: The proposed framework effectively leverages unlabeled data through collaborative pseudo-labeling to overcome low-resource challenges in offensive content detection.

Abstract: Accurate detection of offensive content on social media demands high-quality labeled data; however, such data is often scarce due to the low prevalence of offensive instances and the high cost of manual annotation. To address this low-resource challenge, we propose a self-training framework that leverages abundant unlabeled data through collaborative pseudo-labeling. Starting with a lightweight classifier trained on limited labeled data, our method iteratively assigns pseudo-labels to unlabeled instances with the support of Multi-Agent Vision-Language Models (MA-VLMs). Un-labeled data on which the classifier and MA-VLMs agree are designated as the Agreed-Unknown set, while conflicting samples form the Disagreed-Unknown set. To enhance label reliability, MA-VLMs simulate dual perspectives, moderator and user, capturing both regulatory and subjective viewpoints. The classifier is optimized using a novel Positive-Negative-Unlabeled (PNU) loss, which jointly exploits labeled, Agreed-Unknown, and Disagreed-Unknown data while mitigating pseudo-label noise. Experiments on benchmark datasets demonstrate that our framework substantially outperforms baselines under limited supervision and approaches the performance of large-scale models

Method: Regularized Schrödinger Bridge (RSB) with novel regularized training that perturbs both input states and targets, exposing model to simulated prediction errors and using posterior mean interpolation.

Result: Outperforms state-of-the-art methods on speech enhancement tasks, significantly improving distortion metrics and effectively reducing exposure bias.

Conclusion: RSB effectively addresses key limitations of diffusion models in inverse problems through its regularized training approach.

Abstract: Diffusion models serve as a powerful generative framework for solving inverse problems. However, they still face two key challenges: 1) the distortion-perception tradeoff, where improving perceptual quality often degrades reconstruction fidelity, and 2) the exposure bias problem, where the training-inference input mismatch leads to prediction error accumulation and reduced reconstruction quality. In this work, we propose the Regularized Schrödinger Bridge (RSB), an adaptation of Schrödinger Bridge tailored for inverse problems that addresses the above limitations. RSB employs a novel regularized training strategy that perturbs both the input states and targets, effectively mitigating exposure bias by exposing the model to simulated prediction errors and also alleviating distortion by well-designed interpolation via the posterior mean. Extensive experiments on two typical inverse problems for speech enhancement demonstrate that RSB outperforms state-of-the-art methods, significantly improving distortion metrics and effectively reducing exposure bias.

Method: Proposes MoETTA framework integrating Mixture-of-Experts architecture with structurally decoupled experts to enable adaptation along diverse gradient directions, allowing flexible and disentangled parameter updates for heterogeneous shifts.

Result: Extensive experiments across three mixed distribution shift settings show MoETTA consistently outperforms strong baselines, establishing state-of-the-art performance and demonstrating the benefit of modeling multiple adaptation directions via expert-level diversity.

Conclusion: MoETTA effectively addresses the limitations of existing TTA methods in handling mixed distribution shifts by enabling diverse adaptation paths through expert-level diversity, with new benchmarks capturing realistic deployment challenges including natural, artistic, and adversarial distortions.

Abstract: Test-Time adaptation (TTA) has proven effective in mitigating performance drops under single-domain distribution shifts by updating model parameters during inference. However, real-world deployments often involve mixed distribution shifts, where test samples are affected by diverse and potentially conflicting domain factors, posing significant challenges even for SOTA TTA methods. A key limitation in existing approaches is their reliance on a unified adaptation path, which fails to account for the fact that optimal gradient directions can vary significantly across different domains. Moreover, current benchmarks focus only on synthetic or homogeneous shifts, failing to capture the complexity of real-world heterogeneous mixed distribution shifts. To address this, we propose MoETTA, a novel entropy-based TTA framework that integrates the Mixture-of-Experts (MoE) architecture. Rather than enforcing a single parameter update rule for all test samples, MoETTA introduces a set of structurally decoupled experts, enabling adaptation along diverse gradient directions. This design allows the model to better accommodate heterogeneous shifts through flexible and disentangled parameter updates. To simulate realistic deployment conditions, we introduce two new benchmarks: potpourri and potpourri+. While classical settings focus solely on synthetic corruptions, potpourri encompasses a broader range of domain shifts–including natural, artistic, and adversarial distortions–capturing more realistic deployment challenges. Additionally, potpourri+ further includes source-domain samples to evaluate robustness against catastrophic forgetting. Extensive experiments across three mixed distribution shifts settings show that MoETTA consistently outperforms strong baselines, establishing SOTA performance and highlighting the benefit of modeling multiple adaptation directions via expert-level diversity.

Method: Adapted existing class incremental learning methods to mitigate forgetting in gene incremental learning, establishing a pipeline and evaluations for single-cell transcriptomics data.

Result: Demonstrated the soundness of the framework design and evaluations, and showed effectiveness of method adaptations in mitigating gene forgetting.

Conclusion: Provides a complete benchmark for gene incremental learning in single-cell transcriptomics, addressing a significant research gap in token-based incremental learning.

Abstract: Classes, as fundamental elements of Computer Vision, have been extensively studied within incremental learning frameworks. In contrast, tokens, which play essential roles in many research fields, exhibit similar characteristics of growth, yet investigations into their incremental learning remain significantly scarce. This research gap primarily stems from the holistic nature of tokens in language, which imposes significant challenges on the design of incremental learning frameworks for them. To overcome this obstacle, in this work, we turn to a type of token, gene, for a large-scale biological dataset–single-cell transcriptomics–to formulate a pipeline for gene incremental learning and establish corresponding evaluations. We found that the forgetting problem also exists in gene incremental learning, thus we adapted existing class incremental learning methods to mitigate the forgetting of genes. Through extensive experiments, we demonstrated the soundness of our framework design and evaluations, as well as the effectiveness of our method adaptations. Finally, we provide a complete benchmark for gene incremental learning in single-cell transcriptomics.

Method: Leverages MLIR dialects to auto-generate near-optimal code by composing nanokernels from low-level IR constructs with optimal register utilization, implemented in an MLIR-based compiler supporting vector and tile-based CPU instructions.

Result: Generated nanokernels achieve production-quality performance and are competitive with state-of-the-art microkernel libraries.

Conclusion: The approach successfully bridges the domain-hardware gap by enabling automatic generation of efficient microkernels without requiring low-level library dependencies.

Abstract: The rapidly evolving landscape of AI and machine learning workloads has widened the gap between high-level domain operations and efficient hardware utilization. Achieving near-peak performance still demands deep hardware expertise-experts either handcraft target-specific kernels (e.g., DeepSeek) or rely on specialized libraries (e.g., CUTLASS)-both of which add complexity and limit scalability for most ML practitioners. This paper introduces a compilation scheme that automatically generates scalable, high-performance microkernels by leveraging the MLIR dialects to bridge domain-level operations and processor capabilities. Our approach removes dependence on low-level libraries by enabling the compiler to auto-generate near-optimal code directly. At its core is a mechanism for composing nanokernels from low-level IR constructs with near-optimal register utilization, forming efficient microkernels tailored to each target. We implement this technique in an MLIR-based compiler supporting both vector and tile based CPU instructions. Experiments show that the generated nanokernels are of production-quality, and competitive with state-of-the-art microkernel libraries.

Method: PROF uses LLMs to generate reward function codes and introduces Reward Preference Ranking (RPR) to assess reward quality without environment interactions. It alternates between RPR and text-based gradient optimization to automate reward function selection and refinement.

Result: Empirical results on D4RL show PROF surpasses or matches recent strong baselines across multiple datasets and domains, demonstrating superior performance.

Conclusion: PROF provides an effective automated approach for reward function generation and refinement in offline imitation learning, leveraging LLMs to overcome limitations of traditional similarity-based methods.

Abstract: Offline imitation learning (offline IL) enables training effective policies without requiring explicit reward annotations. Recent approaches attempt to estimate rewards for unlabeled datasets using a small set of expert demonstrations. However, these methods often assume that the similarity between a trajectory and an expert demonstration is positively correlated with the reward, which oversimplifies the underlying reward structure. We propose PROF, a novel framework that leverages large language models (LLMs) to generate and improve executable reward function codes from natural language descriptions and a single expert trajectory. We propose Reward Preference Ranking (RPR), a novel reward function quality assessment and ranking strategy without requiring environment interactions or RL training. RPR calculates the dominance scores of the reward functions, where higher scores indicate better alignment with expert preferences. By alternating between RPR and text-based gradient optimization, PROF fully automates the selection and refinement of optimal reward functions for downstream policy learning. Empirical results on D4RL demonstrate that PROF surpasses or matches recent strong baselines across numerous datasets and domains, highlighting the effectiveness of our approach.

Method: Propose credal ensemble distillation (CED) framework that compresses deep ensembles into a single CREDIT model, which predicts class-wise probability intervals defining credal sets instead of single softmax distributions.

Result: Empirical results show CED achieves superior or comparable uncertainty estimation on out-of-distribution detection benchmarks while substantially reducing inference overhead compared to deep ensembles.

Conclusion: CED effectively addresses the computational limitations of deep ensembles while maintaining strong uncertainty quantification capabilities.

Abstract: Deep ensembles (DE) have emerged as a powerful approach for quantifying predictive uncertainty and distinguishing its aleatoric and epistemic components, thereby enhancing model robustness and reliability. However, their high computational and memory costs during inference pose significant challenges for wide practical deployment. To overcome this issue, we propose credal ensemble distillation (CED), a novel framework that compresses a DE into a single model, CREDIT, for classification tasks. Instead of a single softmax probability distribution, CREDIT predicts class-wise probability intervals that define a credal set, a convex set of probability distributions, for uncertainty quantification. Empirical results on out-of-distribution detection benchmarks demonstrate that CED achieves superior or comparable uncertainty estimation compared to several existing baselines, while substantially reducing inference overhead compared to DE.

Method: DTS adjusts temperature dynamically based on the cross-entropy loss gap between teacher and student distributions, integrating with existing KD frameworks.

Result: DTS consistently outperforms static-temperature baselines across multiple KD strategies on vision (CIFAR-100, Tiny-ImageNet) and NLP tasks (GLUE, Dolly, SelfIns, UnNI, S-NI).

Conclusion: Dynamic temperature scheduling based on teacher-student divergence is effective and can be seamlessly integrated into existing knowledge distillation frameworks.

Abstract: Knowledge Distillation (KD) trains a smaller student model using a large, pre-trained teacher model, with temperature as a key hyperparameter controlling the softness of output probabilities. Traditional methods use a fixed temperature throughout training, which is suboptimal. Moreover, architectural differences between teacher and student often result in mismatched logit magnitudes. We demonstrate that students benefit from softer probabilities early in training but require sharper probabilities in later stages. We introduce Dynamic Temperature Scheduler (DTS), which adjusts temperature dynamically based on the cross-entropy loss gap between teacher and student. To our knowledge, this is the first temperature scheduling method that adapts based on the divergence between teacher and student distributions. Our method integrates seamlessly with existing KD frameworks. We validate DTS across multiple KD strategies on vision (CIFAR-100, Tiny-ImageNet) and NLP tasks (GLUE, Dolly, SelfIns, UnNI, S-NI), consistently outperforming static-temperature baselines. Code is available at https://github.com/Sibgat-Ul/DTS.

Method: Developed Cajal - a typed, higher-order linear programming language that compiles programs to linear neurons, making discrete algorithms differentiable and compatible with gradient-based learning.

Result: Experiments showed that linking Cajal-generated linear neurons with other neural networks enables faster learning, greater data efficiency, and easier debugging by determining part of the network’s function prior to learning.

Conclusion: Linear programming languages like Cajal provide a path to directly program neural networks, enabling rich interplay between learning and discrete programming structures.

Abstract: We don’t program neural networks directly. Instead, we rely on an indirect style where learning algorithms, like gradient descent, determine a neural network’s function by learning from data. This indirect style is often a virtue; it empowers us to solve problems that were previously impossible. But it lacks discrete structure. We can’t compile most algorithms into a neural network – even if these algorithms could help the network learn. This limitation occurs because discrete algorithms are not obviously differentiable, making them incompatible with the gradient-based learning algorithms that determine a neural network’s function. To address this, we introduce $\textsf{Cajal}$: a typed, higher-order and linear programming language intended to be a minimal vehicle for exploring a direct style of programming neural networks. We prove $\textsf{Cajal}$ programs compile to linear neurons, allowing discrete algorithms to be expressed in a differentiable form compatible with gradient-based learning. With our implementation of $\textsf{Cajal}$, we conduct several experiments where we link these linear neurons against other neural networks to determine part of their function prior to learning. Linking with these neurons allows networks to learn faster, with greater data-efficiency, and in a way that’s easier to debug. A key lesson is that linear programming languages provide a path towards directly programming neural networks, enabling a rich interplay between learning and the discrete structures of ordinary programming.

Method: Starting from the co-occurrence matrix underlying GloVe embeddings, demonstrate how projection captures contextual influence, with query-key-value as asymmetric extension for directional relationships. Positional encodings and multi-head attention are structured refinements of this projection principle.

Result: The analysis shows that self-attention emerges from projecting corpus-level co-occurrence statistics into sequence context, with the Transformer’s algebraic form following from these projection principles.

Conclusion: The Transformer architecture’s particular algebraic form follows from projection principles of distributional semantics rather than being an arbitrary design choice, providing a mathematical foundation for self-attention mechanisms.

Abstract: This paper presents a mathematical interpretation of self-attention by connecting it to distributional semantics principles. We show that self-attention emerges from projecting corpus-level co-occurrence statistics into sequence context. Starting from the co-occurrence matrix underlying GloVe embeddings, we demonstrate how the projection naturally captures contextual influence, with the query-key-value mechanism arising as the natural asymmetric extension for modeling directional relationships. Positional encodings and multi-head attention then follow as structured refinements of this same projection principle. Our analysis demonstrates that the Transformer architecture’s particular algebraic form follows from these projection principles rather than being an arbitrary design choice.

Method: Proposes TC² method that splits batches into multiple SSL tasks, uses factor and weight extraction networks with data reconstruction, orthogonality, and sparsity constraints, and employs a two-stage bi-level optimization framework.

Result: Experimental results show consistent improvement in SSL model transferability across multiple downstream tasks.

Conclusion: The proposed TC² method effectively addresses task conflicts and enhances representation transferability in self-supervised learning.

Abstract: In this paper, we explore the transferability of SSL by addressing two central questions: (i) what is the representation transferability of SSL, and (ii) how can we effectively model this transferability? Transferability is defined as the ability of a representation learned from one task to support the objective of another. Inspired by the meta-learning paradigm, we construct multiple SSL tasks within each training batch to support explicitly modeling transferability. Based on empirical evidence and causal analysis, we find that although introducing task-level information improves transferability, it is still hindered by task conflict. To address this issue, we propose a Task Conflict Calibration (TC$^2$) method to alleviate the impact of task conflict. Specifically, it first splits batches to create multiple SSL tasks, infusing task-level information. Next, it uses a factor extraction network to produce causal generative factors for all tasks and a weight extraction network to assign dedicated weights to each sample, employing data reconstruction, orthogonality, and sparsity to ensure effectiveness. Finally, TC$^2$ calibrates sample representations during SSL training and integrates into the pipeline via a two-stage bi-level optimization framework to boost the transferability of learned representations. Experimental results on multiple downstream tasks demonstrate that our method consistently improves the transferability of SSL models.

Method: Introduces ScoresActivation function - a feature-ranking mechanism embedded within the learning pipeline that enables models to prioritize features according to their contribution to predictive performance in a differentiable and end-to-end trainable manner.

Result: Achieves globally faithful feature rankings aligned with SHAP values and ground-truth, 150x faster than SHAP (2s vs 300s), improves classification accuracy by 11.24-29.33%, and demonstrates robustness to irrelevant inputs.

Conclusion: Bridges the gap between model accuracy and interpretability, offering a scalable framework for inherently explainable machine learning that integrates feature importance directly into training.

Abstract: Understanding the decision of large deep learning models is a critical challenge for building transparent and trustworthy systems. Although the current post hoc explanation methods offer valuable insights into feature importance, they are inherently disconnected from the model training process, limiting their faithfulness and utility. In this work, we introduce a novel differentiable approach to global explainability by design, integrating feature importance estimation directly into model training. Central to our method is the ScoresActivation function, a feature-ranking mechanism embedded within the learning pipeline. This integration enables models to prioritize features according to their contribution to predictive performance in a differentiable and end-to-end trainable manner. Evaluations across benchmark datasets show that our approach yields globally faithful, stable feature rankings aligned with SHAP values and ground-truth feature importance, while maintaining high predictive performance. Moreover, feature scoring is 150 times faster than the classical SHAP method, requiring only 2 seconds during training compared to SHAP’s 300 seconds for feature ranking in the same configuration. Our method also improves classification accuracy by 11.24% with 10 features (5 relevant) and 29.33% with 16 features (5 relevant, 11 irrelevant), demonstrating robustness to irrelevant inputs. This work bridges the gap between model accuracy and interpretability, offering a scalable framework for inherently explainable machine learning.

Method: Proposes DAS framework with two key components: (1) adaptive nonparametric drafter built from recent rollouts using incrementally maintained suffix tree, (2) length-aware speculation policy that allocates more aggressive draft budgets to long trajectories dominating makespan.

Result: Experiments on math and code reasoning tasks show up to 50% reduction in rollout time while preserving identical training curves and model outputs.

Conclusion: Distribution-aware speculative decoding can significantly accelerate RL post-training without compromising learning quality, making RL alignment more efficient.

Abstract: Reinforcement learning(RL) post-training has become essential for aligning large language models (LLMs), yet its efficiency is increasingly constrained by the rollout phase, where long trajectories are generated token by token. We identify a major bottleneck:the long-tail distribution of rollout lengths, where a small fraction of long generations dominates wall clock time and a complementary opportunity; the availability of historical rollouts that reveal stable prompt level patterns across training epochs. Motivated by these observations, we propose DAS, a Distribution Aware Speculative decoding framework that accelerates RL rollouts without altering model outputs. DAS integrates two key ideas: an adaptive, nonparametric drafter built from recent rollouts using an incrementally maintained suffix tree, and a length aware speculation policy that allocates more aggressive draft budgets to long trajectories that dominate makespan. This design exploits rollout history to sustain acceptance while balancing base and token level costs during decoding. Experiments on math and code reasoning tasks show that DAS reduces rollout time up to 50% while preserving identical training curves, demonstrating that distribution-aware speculative decoding can significantly accelerate RL post training without compromising learning quality.

Method: AnaCP (Analytic Contrastive Projection) preserves analytic classifier efficiency while enabling incremental feature adaptation without gradient-based training.

Result: AnaCP outperforms existing baselines and achieves the accuracy level of joint training, which is considered the upper bound of CIL.

Conclusion: AnaCP successfully addresses the feature adaptation limitation in CIL while maintaining efficiency and avoiding catastrophic forgetting.

Abstract: This paper studies the problem of class-incremental learning (CIL), a core setting within continual learning where a model learns a sequence of tasks, each containing a distinct set of classes. Traditional CIL methods, which do not leverage pre-trained models (PTMs), suffer from catastrophic forgetting (CF) due to the need to incrementally learn both feature representations and the classifier. The integration of PTMs into CIL has recently led to efficient approaches that treat the PTM as a fixed feature extractor combined with analytic classifiers, achieving state-of-the-art performance. However, they still face a major limitation: the inability to continually adapt feature representations to best suit the CIL tasks, leading to suboptimal performance. To address this, we propose AnaCP (Analytic Contrastive Projection), a novel method that preserves the efficiency of analytic classifiers while enabling incremental feature adaptation without gradient-based training, thereby eliminating the CF caused by gradient updates. Our experiments show that AnaCP not only outperforms existing baselines but also achieves the accuracy level of joint training, which is regarded as the upper bound of CIL.

Method: Using tractable probabilistic models (TPMs), particularly sum-product networks (SPNs), to enable exact and scalable inference of scenario likelihoods, marginals, and conditional probabilities.

Result: The approach allows direct embedding of chance-constrained optimization into investment planning, supporting robust scenario expansion and risk assessment with computational and reliability advantages.

Conclusion: TPMs provide an effective alternative to traditional scenario-based models for power system planning, enabling better uncertainty quantification and robust decision-making.

Abstract: Investment planning in power utilities, such as generation and transmission expansion, requires decade-long forecasts under profound uncertainty. Forecasting of energy mix and energy use decades ahead is nontrivial. Classical approaches focus on generating a finite number of scenarios (modeled as a mixture of Diracs in statistical theory terms), which limits insight into scenario-specific volatility and hinders robust decision-making. We propose an alternative using tractable probabilistic models (TPMs), particularly sum-product networks (SPNs). These models enable exact, scalable inference of key quantities such as scenario likelihoods, marginals, and conditional probabilities, supporting robust scenario expansion and risk assessment. This framework enables direct embedding of chance-constrained optimization into investment planning, enforcing safety or reliability with prescribed confidence levels. TPMs allow both scenario analysis and volatility quantification by compactly representing high-dimensional uncertainties. We demonstrate the approach’s effectiveness through a representative power system planning case study, illustrating computational and reliability advantages over traditional scenario-based models.

Method: Proposes MargNet which incorporates adaptive marginal selection strategy from statistical models into neural networks, training NNs to generate data that conforms to selected marginals.

Result: On sparsely correlated datasets: achieves utility close to best statistical methods with 7x speedup. On densely correlated datasets: establishes new SOTA, reducing fidelity error by up to 26% compared to previous best methods.

Conclusion: MargNet successfully bridges the gap between statistical models and neural networks, demonstrating that neural networks can outperform statistical methods when properly designed for complex dependency structures in DP data synthesis.

Abstract: In differentially private (DP) tabular data synthesis, the consensus is that statistical models are better than neural network (NN)-based methods. However, we argue that this conclusion is incomplete and overlooks the challenge of densely correlated datasets, where intricate dependencies can overwhelm statistical models. In such complex scenarios, neural networks are more suitable due to their capacity to fit complex distributions by learning directly from samples. Despite this potential, existing NN-based algorithms still suffer from significant limitations. We therefore propose MargNet, incorporating successful algorithmic designs of statistical models into neural networks. MargNet applies an adaptive marginal selection strategy and trains the neural networks to generate data that conforms to the selected marginals. On sparsely correlated datasets, our approach achieves utility close to the best statistical method while offering an average 7$\times$ speedup over it. More importantly, on densely correlated datasets, MargNet establishes a new state-of-the-art, reducing fidelity error by up to 26% compared to the previous best. We release our code on GitHub.\footnote{https://github.com/KaiChen9909/margnet}

Method: Hourly weather maps are encoded as spatial tokens using a lightweight CNN, then processed by a transformer to capture temporal dynamics across a 45-hour forecast horizon.

Result: Evaluation against ENTSO-E operational forecasts shows 60% RMSE reduction for wind and 20% for solar forecasting, despite disadvantages in input initialization.

Conclusion: The approach successfully integrates spatial and temporal weather information for improved renewable energy forecasting, with a live dashboard available for daily forecasts.

Abstract: Accurate renewable energy forecasting is essential to reduce dependence on fossil fuels and enabling grid decarbonization. However, current approaches fail to effectively integrate the rich spatial context of weather patterns with their temporal evolution. This work introduces a novel approach that treats weather maps as tokens in transformer sequences to predict renewable energy. Hourly weather maps are encoded as spatial tokens using a lightweight convolutional neural network, and then processed by a transformer to capture temporal dynamics across a 45-hour forecast horizon. Despite disadvantages in input initialization, evaluation against ENTSO-E operational forecasts shows a reduction in RMSE of about 60% and 20% for wind and solar respectively. A live dashboard showing daily forecasts is available at: https://www.sardiniaforecast.ifabfoundation.it.

Method: Proposes Complex-Weighted Convolutional Network (CWCN) that assigns complex numbers to edges, extending random walks into complex domain with learnable matrices and nonlinear activations.

Result: CWCN achieves competitive performance on benchmark datasets, requires no additional hyperparameters beyond standard GNNs, and is proven to solve any node-classification task in steady state with appropriate complex weights.

Conclusion: Complex-weighted diffusion provides a principled and general mechanism for enhancing GNN expressiveness, offering theoretically grounded and practically effective models.

Abstract: Graph Neural Networks (GNNs) have achieved remarkable success across diverse applications, yet they remain limited by oversmoothing and poor performance on heterophilic graphs. To address these challenges, we introduce a novel framework that equips graphs with a complex-weighted structure, assigning each edge a complex number to drive a diffusion process that extends random walks into the complex domain. We prove that this diffusion is highly expressive: with appropriately chosen complex weights, any node-classification task can be solved in the steady state of a complex random walk. Building on this insight, we propose the Complex-Weighted Convolutional Network (CWCN), which learns suitable complex-weighted structures directly from data while enriching diffusion with learnable matrices and nonlinear activations. CWCN is simple to implement, requires no additional hyperparameters beyond those of standard GNNs, and achieves competitive performance on benchmark datasets. Our results demonstrate that complex-weighted diffusion provides a principled and general mechanism for enhancing GNN expressiveness, opening new avenues for models that are both theoretically grounded and practically effective.

Method: Systematic examination of BR from 0.2 to 5.0 across 39 regression datasets and 16 RF configurations using repeated two-fold cross-validation and MSE evaluation, plus experiments on synthetic datasets with controlled noise.

Result: Tuning BR yields significant improvements: BR ≤ 1.0 optimal for 24 datasets, BR > 1.0 for 15, BR = 1.0 optimal in only 4 cases. Dataset characteristics predict optimal BR - strong global relationships favor higher BRs, high local variance favors lower BRs.

Conclusion: Bootstrap rate is an influential hyperparameter that should be tuned for optimal RF regression performance, with clear bias-variance trade-off relationships observed across different noise levels.

Abstract: Random Forests (RFs) typically train each tree on a bootstrap sample of the same size as the training set, i.e., bootstrap rate (BR) equals 1.0. We systematically examine how varying BR from 0.2 to 5.0 affects RF performance across 39 heterogeneous regression datasets and 16 RF configurations, evaluating with repeated two-fold cross-validation and mean squared error. Our results demonstrate that tuning the BR can yield significant improvements over the default: the best setup relied on BR \leq 1.0 for 24 datasets, BR > 1.0 for 15, and BR = 1.0 was optimal in 4 cases only. We establish a link between dataset characteristics and the preferred BR: datasets with strong global feature-target relationships favor higher BRs, while those with higher local target variance benefit from lower BRs. To further investigate this relationship, we conducted experiments on synthetic datasets with controlled noise levels. These experiments reproduce the observed bias-variance trade-off: in low-noise scenarios, higher BRs effectively reduce model bias, whereas in high-noise settings, lower BRs help reduce model variance. Overall, BR is an influential hyperparameter that should be tuned to optimize RF regression models.

Method: Proves generalization error bounds for SNNs using Wasserstein-distance approach, improves previous training methods, and validates through numerical experiments on multidimensional uncertainty quantification tasks.

Result: When noise is heterogeneous across dimensions, convergence rate of generalization error may not explicitly depend on model dimensionality, partially alleviating the curse of dimensionality. The improved Wasserstein-distance approach successfully trains SNNs for multidimensional uncertainty models.

Conclusion: The Wasserstein-distance approach enables effective training of stochastic neural networks for multidimensional random field reconstruction, with improved scalability and robustness against the curse of dimensionality in certain noise conditions.

Abstract: In this paper, we analyze the scalability of a recent Wasserstein-distance approach for training stochastic neural networks (SNNs) to reconstruct multidimensional random field models. We prove a generalization error bound for reconstructing multidimensional random field models on training stochastic neural networks with a limited number of training data. Our results indicate that when noise is heterogeneous across dimensions, the convergence rate of the generalization error may not depend explicitly on the model’s dimensionality, partially alleviating the “curse of dimensionality” for learning multidimensional random field models from a finite number of data points. Additionally, we improve the previous Wasserstein-distance SNN training approach and showcase the robustness of the SNN. Through numerical experiments on different multidimensional uncertainty quantification tasks, we show that our Wasserstein-distance approach can successfully train stochastic neural networks to learn multidimensional uncertainty models.

Method: Apply PCA Whitening to input activations, evaluate ReLU and Top-K SAEs across various model architectures, widths, and sparsity regimes using SAEBench benchmark.

Result: Whitening consistently improves interpretability metrics (sparse probing accuracy, feature disentanglement) while slightly reducing reconstruction quality.

Conclusion: Whitening should be a default preprocessing step for SAE training when interpretability is prioritized over perfect reconstruction, challenging the sparsity-fidelity trade-off assumption.

Abstract: Sparse autoencoders (SAEs) have emerged as a promising approach for learning interpretable features from neural network activations. However, the optimization landscape for SAE training can be challenging due to correlations in the input data. We demonstrate that applying PCA Whitening to input activations – a standard preprocessing technique in classical sparse coding – improves SAE performance across multiple metrics. Through theoretical analysis and simulation, we show that whitening transforms the optimization landscape, making it more convex and easier to navigate. We evaluate both ReLU and Top-K SAEs across diverse model architectures, widths, and sparsity regimes. Empirical evaluation on SAEBench, a comprehensive benchmark for sparse autoencoders, reveals that whitening consistently improves interpretability metrics, including sparse probing accuracy and feature disentanglement, despite minor drops in reconstruction quality. Our results challenge the assumption that interpretability aligns with an optimal sparsity–fidelity trade-off and suggest that whitening should be considered as a default preprocessing step for SAE training, particularly when interpretability is prioritized over perfect reconstruction.

Method: Two-stage approach: 1) Semantic labeling algorithm that assigns node-level correctness without penalizing structural containers or alias variations, 2) Supervised classifier using schema-aware features (identifier validity, alias resolution, type compatibility, scope ambiguity, typo signals) to predict per-node error probabilities.

Result: Substantially outperforms token log-probabilities with +27.44% average AUC improvement across multiple databases and datasets, while maintaining robustness in cross-database evaluation.

Conclusion: Node-centric, semantically grounded uncertainty estimation provides a strong and interpretable alternative to aggregate sequence-level confidence measures, enabling targeted repair, human-in-the-loop review, and selective execution.

Abstract: We present a practical framework for detecting errors in LLM-generated SQL by estimating uncertainty at the level of individual nodes in the query’s abstract syntax tree (AST). Our approach proceeds in two stages. First, we introduce a semantically aware labeling algorithm that, given a generated SQL and a gold reference, assigns node-level correctness without over-penalizing structural containers or alias variation. Second, we represent each node with a rich set of schema-aware and lexical features - capturing identifier validity, alias resolution, type compatibility, ambiguity in scope, and typo signals - and train a supervised classifier to predict per-node error probabilities. We interpret these probabilities as calibrated uncertainty, enabling fine-grained diagnostics that pinpoint exactly where a query is likely to be wrong. Across multiple databases and datasets, our method substantially outperforms token log-probabilities: average AUC improves by +27.44% while maintaining robustness under cross-database evaluation. Beyond serving as an accuracy signal, node-level uncertainty supports targeted repair, human-in-the-loop review, and downstream selective execution. Together, these results establish node-centric, semantically grounded uncertainty estimation as a strong and interpretable alternative to aggregate sequence level confidence measures.

Method: Theoretical analysis establishing lower bounds on first-order oracle queries for differentially private empirical/population risk minimization, considering both non-smooth and smooth Lipschitz convex losses under different oracle models (private proxy oracle vs. private optimizer).

Result: Proved tight lower bounds: Ω(min{√d/α², d/log(1/α)}) for non-smooth losses with private proxy oracle, Ω(d/(m̄α²)) for minibatch algorithms, and Ω̃(√d/α + min{1/α², n}) for smooth losses with private optimizer. Also showed Ω(d/(α²Γ)) bounds for information-limited oracles.

Conclusion: Differentially private optimization incurs dimension-dependent runtime penalties, and gradient quantization techniques have fundamental limitations due to information constraints.

Abstract: We study the running time, in terms of first order oracle queries, of differentially private empirical/population risk minimization of Lipschitz convex losses. We first consider the setting where the loss is non-smooth and the optimizer interacts with a private proxy oracle, which sends only private messages about a minibatch of gradients. In this setting, we show that expected running time $Ω(\min{\frac{\sqrt{d}}{α^2}, \frac{d}{\log(1/α)}})$ is necessary to achieve $α$ excess risk on problems of dimension $d$ when $d \geq 1/α^2$. Upper bounds via DP-SGD show these results are tight when $d>\tildeΩ(1/α^4)$. We further show our lower bound can be strengthened to $Ω(\min{\frac{d}{\bar{m}α^2}, \frac{d}{\log(1/α)} })$ for algorithms which use minibatches of size at most $\bar{m} < \sqrt{d}$. We next consider smooth losses, where we relax the private oracle assumption and give lower bounds under only the condition that the optimizer is private. Here, we lower bound the expected number of first order oracle calls by $\tildeΩ\big(\frac{\sqrt{d}}α + \min{\frac{1}{α^2}, n}\big)$, where $n$ is the size of the dataset. Modifications to existing algorithms show this bound is nearly tight. Compared to non-private lower bounds, our results show that differentially private optimizers pay a dimension dependent runtime penalty. Finally, as a natural extension of our proof technique, we show lower bounds in the non-smooth setting for optimizers interacting with information limited oracles. Specifically, if the proxy oracle transmits at most $Γ$-bits of information about the gradients in the minibatch, then $Ω\big(\min{\frac{d}{α^2Γ}, \frac{d}{\log(1/α)}}\big)$ oracle calls are needed. This result shows fundamental limitations of gradient quantization techniques in optimization.

Method: Utilizes tractable probabilistic circuits to learn distributions and answer marginal queries, circumventing the need for dynamic programming restrictions on parent sets.

Result: Empirical results show that using probabilistic circuits for marginalization improves Bayesian structure learner performance compared to current methods.

Conclusion: Probabilistic circuits provide an effective alternative to dynamic programming for marginalization in Bayesian structure learning, enabling better performance without parent set restrictions.

Abstract: Bayesian networks (BNs) are a widely used class of probabilistic graphical models employed in numerous application domains. However, inferring the network’s graphical structure from data remains challenging. Bayesian structure learners approach this problem by inferring a posterior distribution over the possible directed acyclic graphs underlying the BN. The inference process often requires marginalizing over probability distributions, which is typically done using dynamic programming methods that restrict the set of possible parents for each node. Instead, we present a novel method that utilizes tractable probabilistic circuits to circumvent this restriction. This method utilizes a new learning routine that trains these circuits on both the original distribution and marginal queries. The architecture of probabilistic circuits then inherently allows for fast and exact marginalization on the learned distribution. We then show empirically that utilizing our method to answer marginals allows Bayesian structure learners to improve their performance compared to current methods.

Method: Uses region-focused adversarial examples to craft imperceptible perturbations that spoof certificates and achieve certification radii larger than the source class ghost certificates, evaluated extensively on ImageNet.

Result: Successfully bypasses state-of-the-art certified defenses like Densepure, demonstrating the ability to generate deceptive robustness guarantees for adversarial inputs.

Conclusion: Highlights the need to better understand the limits of robustness certification methods and reveals vulnerabilities in current certified defense frameworks.

Abstract: Certified defenses promise provable robustness guarantees. We study the malicious exploitation of probabilistic certification frameworks to better understand the limits of guarantee provisions. Now, the objective is to not only mislead a classifier, but also manipulate the certification process to generate a robustness guarantee for an adversarial input certificate spoofing. A recent study in ICLR demonstrated that crafting large perturbations can shift inputs far into regions capable of generating a certificate for an incorrect class. Our study investigates if perturbations needed to cause a misclassification and yet coax a certified model into issuing a deceptive, large robustness radius for a target class can still be made small and imperceptible. We explore the idea of region-focused adversarial examples to craft imperceptible perturbations, spoof certificates and achieve certification radii larger than the source class ghost certificates. Extensive evaluations with the ImageNet demonstrate the ability to effectively bypass state-of-the-art certified defenses such as Densepure. Our work underscores the need to better understand the limits of robustness certification methods.

Method: Performed refusal unlearning on Cybersecurity and Safety concepts, evaluated EMA across seven RAI domains, analyzed concept entanglements via representation-level concept vectors, and tested restoration using cross-entropy loss on retain data.

Result: Safety concept unlearning caused larger EMA impact across unrelated domains like bias. EMA was consistent across Mistral-7b-0.3v and Qwen-7b-2.5 models. Cross-entropy loss on retain data largely restored alignment while maintaining low refusal rates on target concepts.

Conclusion: Concepts with higher representation similarity in earlier layers are more susceptible to EMA after refusal unlearning interventions, highlighting the need for careful consideration of concept entanglements in targeted unlearning approaches.

Abstract: Recent work has shown that fine-tuning on insecure code data can trigger an emergent misalignment (EMA) phenomenon, where models generate malicious responses even to prompts unrelated to the original insecure code-writing task. Such cross-domain generalization of harmful behavior underscores the need for a deeper understanding of the algorithms, tasks, and datasets that induce emergent misalignment. In this work, we extend this study by demonstrating that emergent misalignment can also arise from narrow refusal unlearning in specific domains. We perform refusal unlearning on Cybersecurity and Safety concept, and evaluate EMA by monitoring refusal scores across seven responsible AI (RAI) domains, Cybersecurity, Safety, Toxicity, Bias, Sensitive Content, Medical/Legal, and Privacy. Our work shows that narrow domain unlearning can yield compliance responses for the targeted concept, however, it may also propagate EMA to unrelated domains. Among the two intervened concepts, Cybersecurity and Safety, we find that the safety concept can have larger EMA impact, i.e, causing lower refusal scores, across other unrelated domains such as bias. We observe this effect consistently across two model families, Mistral-7b-0.3v, and Qwen-7b-2.5. Further, we show that refusal unlearning augmented with cross-entropy loss function on a small set of retain data from the affected domains can largely, if not fully, restore alignment across the impacted domains while having lower refusal rate on the concept we perform unlearning on. To investigate the underlying causes of EMA, we analyze concept entanglements at the representation level via concept vectors. Our analysis reveals that concepts with higher representation similarity in earlier layers are more susceptible to EMA after intervention when the refusal stream is altered through targeted refusal unlearning.

Method: Three-layer architecture: 1) Transfer learning from 22,673 public records using SHAP-based knowledge transfer, 2) Hierarchical Bayesian modeling with adaptive shrinkage across multiple SMEs, 3) Conformal prediction for uncertainty quantification with coverage guarantees.

Result: 96.7% ±4.2% AUC with 100 observations per business (+24.2 points improvement), 92% empirical coverage at 90% target, 33-minute training on standard CPU hardware.

Conclusion: SmallML democratizes AI for 33 million US SMEs previously excluded from machine learning by enabling enterprise-grade predictions with minimal data requirements.

Abstract: Small and medium-sized enterprises (SMEs) represent 99.9% of U.S. businesses yet remain systematically excluded from AI due to a mismatch between their operational scale and modern machine learning’s data requirements. This paper introduces SmallML, a Bayesian transfer learning framework achieving enterprise-level prediction accuracy with datasets as small as 50-200 observations. We develop a three-layer architecture integrating transfer learning, hierarchical Bayesian modeling, and conformal prediction. Layer 1 extracts informative priors from 22,673 public records using a SHAP-based procedure transferring knowledge from gradient boosting to logistic regression. Layer 2 implements hierarchical pooling across J=5-50 SMEs with adaptive shrinkage, balancing population patterns with entity-specific characteristics. Layer 3 provides conformal sets with finite-sample coverage guarantees P(y in C(x)) >= 1-alpha for distribution-free uncertainty quantification. Validation on customer churn data demonstrates 96.7% +/- 4.2% AUC with 100 observations per business – a +24.2 point improvement over independent logistic regression (72.5% +/- 8.1%), with p < 0.000001. Conformal prediction achieves 92% empirical coverage at 90% target. Training completes in 33 minutes on standard CPU hardware. By enabling enterprise-grade predictions for 33 million U.S. SMEs previously excluded from machine learning, SmallML addresses a critical gap in AI democratization. Keywords: Bayesian transfer learning, hierarchical models, conformal prediction, small-data analytics, SME machine learning

Method: RC selects base density in tangent space so likelihood depends only on geodesic distance, decoupling parameter semantics from curvature. Introduces Balanced-Exponential (bExp) chart family that balances volume distortion and geodesic error while preserving manifold density and Fisher information.

Result: RC yields stable generative models across various applications (VAEs, flows on images/graphs, protein models), improves likelihoods, restores clean geodesic radii, and prevents radius blow-ups in high-dimensional flows.

Conclusion: RC-bExp provides a robust default for likelihood-trained generative models on manifolds by decoupling curvature from parameters and enabling numerical preconditioning of charts.

Abstract: Generative models on curved spaces rely on charts to map Euclidean spaces to manifolds. Exponential maps preserve geodesics but have stiff, radius-dependent Jacobians, while volume-preserving charts maintain densities but distort geodesic distances. Both approaches entangle curvature with model parameters, inflating gradient variance. In high-dimensional latent normalizing flows, the wrapped exponential prior can stretch radii far beyond the curvature scale, leading to poor test likelihoods and stiff solvers. We introduce Radial Compensation (RC), an information-geometric method that selects the base density in the tangent space so that the likelihood depends only on geodesic distance from a pole, decoupling parameter semantics from curvature. RC lets radial parameters retain their usual meaning in geodesic units, while the chart can be tuned as a numerical preconditioner. We extend RC to manifolds with known geodesic polar volume and show that RC is the only construction for geodesic-radial likelihoods with curvature-invariant Fisher information. We derive the Balanced-Exponential (bExp) chart family, balancing volume distortion and geodesic error. Under RC, all bExp settings preserve the same manifold density and Fisher information, with smaller dial values reducing gradient variance and flow cost. Empirically, RC yields stable generative models across densities, VAEs, flows on images and graphs, and protein models. RC improves likelihoods, restores clean geodesic radii, and prevents radius blow-ups in high-dimensional flows, making RC-bExp a robust default for likelihood-trained generative models on manifolds.

Method: Multimodal framework integrating wearable sensor data (accelerometer and PPG) with novel SmoothDiff feature for motion recognition using LSTM, and HRV features from PPG for stress estimation using MLP classifier.

Result: 96.8% accuracy and 95.9% F1-score for motion phase recognition; 80% accuracy for stress level classification between high and low stress.

Conclusion: Integration of motion and physiological sensing provides meaningful insights into athletes’ technical and mental states, offering foundation for real-time feedback systems in precision sports.

Abstract: In precision sports such as archery, athletes’ performance depends on both biomechanical stability and psychological resilience. Traditional motion analysis systems are often expensive and intrusive, limiting their use in natural training environments. To address this limitation, we propose a machine learning-based multimodal framework that integrates wearable sensor data for simultaneous action recognition and stress estimation. Using a self-developed wrist-worn device equipped with an accelerometer and photoplethysmography (PPG) sensor, we collected synchronized motion and physiological data during real archery sessions. For motion recognition, we introduce a novel feature–Smoothed Differential Acceleration (SmoothDiff)–and employ a Long Short-Term Memory (LSTM) model to identify motion phases, achieving 96.8% accuracy and 95.9% F1-score. For stress estimation, we extract heart rate variability (HRV) features from PPG signals and apply a Multi-Layer Perceptron (MLP) classifier, achieving 80% accuracy in distinguishing high- and low-stress levels. The proposed framework demonstrates that integrating motion and physiological sensing can provide meaningful insights into athletes’ technical and mental states. This approach offers a foundation for developing intelligent, real-time feedback systems for training optimization in archery and other precision sports.

Method: Proposes CafeMed with two key components: Causal Weight Generator (CWG) that transforms static causal effects into dynamic modulation weights based on patient states, and Channel Harmonized Attention Refinement Module (CHARM) that captures interdependencies between diagnoses and procedures.

Result: Extensive experiments on MIMIC-III and MIMIC-IV datasets show CafeMed significantly outperforms state-of-the-art baselines, achieving superior accuracy in medication prediction while maintaining lower drug-drug interaction rates.

Conclusion: Incorporating dynamic causal relationships and cross-modal synergies leads to more clinically-aligned and personalized medication recommendations.

Abstract: Medication recommendation systems play a crucial role in assisting clinicians with personalized treatment decisions. While existing approaches have made significant progress in learning medication representations, they suffer from two fundamental limitations: (i) treating medical entities as independent features without modeling their synergistic effects on medication selection; (ii) employing static causal relationships that fail to adapt to patient-specific contexts and health states. To address these challenges, we propose CafeMed, a framework that integrates dynamic causal reasoning with cross-modal attention for safe and accurate medication recommendation. CafeMed introduces two key components: the Causal Weight Generator (CWG) that transforms static causal effects into dynamic modulation weights based on individual patient states, and the Channel Harmonized Attention Refinement Module (CHARM) that captures complex interdependencies between diagnoses and procedures. This design enables CafeMed to model how different medical conditions jointly influence treatment decisions while maintaining medication safety constraints. Extensive experiments on MIMIC-III and MIMIC-IV datasets demonstrate that CafeMed significantly outperforms state-of-the-art baselines, achieving superior accuracy in medication prediction while maintaining the lower drug–drug interaction rates. Our results indicate that incorporating dynamic causal relationships and cross-modal synergies leads to more clinically-aligned and personalized medication recommendations. Our code is released publicly at https://github.com/rkl71/CafeMed.

Method: CFG-EC actively realigns unconditional noise error to be orthogonal to conditional error, preventing interference between guidance components and constraining sampling error bounds.

Result: CFG-EC outperforms CFG and CFG++ in handling unconditional components, delivering better performance in low guidance sampling and consistently higher prompt alignment.

Conclusion: CFG-EC provides a versatile correction scheme that establishes more reliable guidance trajectories for high-fidelity image generation across various CFG-based methods.

Abstract: Classifier-Free Guidance (CFG) has become a mainstream approach for simultaneously improving prompt fidelity and generation quality in conditional generative models. During training, CFG stochastically alternates between conditional and null prompts to enable both conditional and unconditional generation. However, during sampling, CFG outputs both null and conditional prompts simultaneously, leading to inconsistent noise estimates between the training and sampling processes. To reduce this error, we propose CFG-EC, a versatile correction scheme augmentable to any CFG-based method by refining the unconditional noise predictions. CFG-EC actively realigns the unconditional noise error component to be orthogonal to the conditional error component. This corrective maneuver prevents interference between the two guidance components, thereby constraining the sampling error’s upper bound and establishing more reliable guidance trajectories for high-fidelity image generation. Our numerical experiments show that CFG-EC handles the unconditional component more effectively than CFG and CFG++, delivering a marked performance increase in the low guidance sampling regime and consistently higher prompt alignment across the board.

Method: Uses fine-grained CSI graph construction with APs as nodes, amplitude-phase fusion for reliable CSI images, dimension-consistent feature extraction, and similarity-guided meta-learning for rapid adaptation to new scenarios.

Result: Extensive experiments show Meta-SimGNN outperforms baseline methods in localization generalization and accuracy across different scenarios using commodity WiFi devices.

Conclusion: The proposed system effectively addresses device configuration variations in WiFi localization through graph neural networks and meta-learning, achieving improved robustness and generalization.

Abstract: To promote the practicality of deep learning-based localization, existing studies aim to address the issue of scenario dependence through meta-learning. However, these studies primarily focus on variations in environmental layouts while overlooking the impact of changes in device configurations, such as bandwidth, the number of access points (APs), and the number of antennas used. Unlike environmental changes, variations in device configurations affect the dimensionality of channel state information (CSI), thereby compromising neural network usability. To address this issue, we propose Meta-SimGNN, a novel WiFi localization system that integrates graph neural networks with meta-learning to improve localization generalization and robustness. First, we introduce a fine-grained CSI graph construction scheme, where each AP is treated as a graph node, allowing for adaptability to changes in the number of APs. To structure the features of each node, we propose an amplitude-phase fusion method and a feature extraction method. The former utilizes both amplitude and phase to construct CSI images, enhancing data reliability, while the latter extracts dimension-consistent features to address variations in bandwidth and the number of antennas. Second, a similarity-guided meta-learning strategy is developed to enhance adaptability in diverse scenarios. The initial model parameters for the fine-tuning stage are determined by comparing the similarity between the new scenario and historical scenarios, facilitating rapid adaptation of the model to the new localization scenario. Extensive experimental results over commodity WiFi devices in different scenarios show that Meta-SimGNN outperforms the baseline methods in terms of localization generalization and accuracy.

Method: Leverages inherent randomness of data distributions to enable privacy evaluation without altering original datasets. Extends privacy auditing beyond traditional membership inference to protected attributes, with labels as a special case.

Result: Experiments on Criteo and CIFAR-10 datasets demonstrate effectiveness in auditing label privacy guarantees.

Conclusion: This work opens new avenues for practical privacy auditing in large-scale production environments by eliminating the need for dataset modifications.

Abstract: Differential privacy (DP) auditing is essential for evaluating privacy guarantees in machine learning systems. Existing auditing methods, however, pose a significant challenge for large-scale systems since they require modifying the training dataset – for instance, by injecting out-of-distribution canaries or removing samples from training. Such interventions on the training data pipeline are resource-intensive and involve considerable engineering overhead. We introduce a novel observational auditing framework that leverages the inherent randomness of data distributions, enabling privacy evaluation without altering the original dataset. Our approach extends privacy auditing beyond traditional membership inference to protected attributes, with labels as a special case, addressing a key gap in existing techniques. We provide theoretical foundations for our method and perform experiments on Criteo and CIFAR-10 datasets that demonstrate its effectiveness in auditing label privacy guarantees. This work opens new avenues for practical privacy auditing in large-scale production environments.

Method: MoE-SpeQ employs a small on-device draft model to predict future token expert sequences, enabling runtime orchestrator to prefetch experts from host memory, overlapping I/O with computation. An adaptive governor using Amortization Roofline Model dynamically tunes speculation strategy.

Result: Evaluation on memory-constrained devices shows MoE-SpeQ achieves up to 2.34x speedup over state-of-the-art offloading framework for Phi-MoE model.

Conclusion: MoE-SpeQ establishes a principled approach for managing data-dependent memory access in resource-limited environments, making MoE inference more accessible on commodity hardware.

Abstract: The immense memory requirements of state-of-the-art Mixture-of-Experts (MoE) models present a significant challenge for inference, often exceeding the capacity of a single accelerator. While offloading experts to host memory is a common solution, it introduces a severe I/O bottleneck over the PCIe bus, as the data-dependent nature of expert selection places these synchronous transfers directly on the critical path of execution, crippling performance. This paper argues that the I/O bottleneck can be overcome by trading a small amount of cheap, on-device computation to hide the immense cost of data movement. We present MoE-SpeQ, a new inference system built on a novel co-design of speculative execution and expert offloading. MoE-SpeQ employs a small, on-device draft model to predict the sequence of required experts for future tokens. This foresight enables a runtime orchestrator to prefetch these experts from host memory, effectively overlapping the expensive I/O with useful computation and hiding the latency from the critical path. To maximize performance, an adaptive governor, guided by an Amortization Roofline Model, dynamically tunes the speculation strategy to the underlying hardware. Our evaluation on memory-constrained devices shows that for the Phi-MoE model, MoE-SpeQ achieves at most 2.34x speedup over the state-of-the-art offloading framework. Our work establishes a new, principled approach for managing data-dependent memory access in resource-limited environments, making MoE inference more accessible on commodity hardware.

Method: Soft-label training that treats annotation distributions as ground truth rather than aggregating them into single labels.

Result: Across vision and NLP tasks, soft-label training achieves 32% lower KL divergence from human annotations, 61% stronger correlation between model and annotation entropy, while matching hard-label training accuracy.

Conclusion: Annotation distributions should be repositioned from noisy signals to faithful representations of epistemic uncertainty that models should learn to reproduce.

Abstract: Many machine learning tasks involve inherent subjectivity, where annotators naturally provide varied labels. Standard practice collapses these label distributions into single labels, aggregating diverse human judgments into point estimates. We argue that this approach is epistemically misaligned for ambiguous data–the annotation distribution itself should be regarded as the ground truth. Training on collapsed single labels forces models to express false confidence on fundamentally ambiguous cases, creating a misalignment between model certainty and the diversity of human perception. We demonstrate empirically that soft-label training, which treats annotation distributions as ground truth, preserves epistemic uncertainty. Across both vision and NLP tasks, soft-label training achieves 32% lower KL divergence from human annotations and 61% stronger correlation between model and annotation entropy, while matching the accuracy of hard-label training. Our work repositions annotation distributions from noisy signals to be aggregated away, to faithful representations of epistemic uncertainty that models should learn to reproduce.

Method: Proposes SSC framework that estimates counterfactual hazard trajectories as weighted combinations of observed trajectories from other units in panel data, leveraging low-rank structure from parametric survival models.

Result: Establishes identification and finite sample guarantees for SSC, validates on multi-country cancer treatment data showing novel therapies associated with improved survival (lower post-intervention hazard trajectories).

Conclusion: SSC provides a general, interpretable tool for counterfactual survival inference across medicine, economics, and public policy using observational data with formal guarantees.

Abstract: Estimating causal effects on time-to-event outcomes from observational data is particularly challenging due to censoring, limited sample sizes, and non-random treatment assignment. The need for answering such “when-if” questions–how the timing of an event would change under a specified intervention–commonly arises in real-world settings with heterogeneous treatment adoption and confounding. To address these challenges, we propose Synthetic Survival Control (SSC) to estimate counterfactual hazard trajectories in a panel data setting where multiple units experience potentially different treatments over multiple periods. In such a setting, SSC estimates the counterfactual hazard trajectory for a unit of interest as a weighted combination of the observed trajectories from other units. To provide formal justification, we introduce a panel framework with a low-rank structure for causal survival analysis. Indeed, such a structure naturally arises under classical parametric survival models. Within this framework, for the causal estimand of interest, we establish identification and finite sample guarantees for SSC. We validate our approach using a multi-country clinical dataset of cancer treatment outcomes, where the staggered introduction of new therapies creates a quasi-experimental setting. Empirically, we find that access to novel treatments is associated with improved survival, as reflected by lower post-intervention hazard trajectories relative to their synthetic counterparts. Given the broad relevance of survival analysis across medicine, economics, and public policy, our framework offers a general and interpretable tool for counterfactual survival inference using observational data.

Method: Used StereoSet and CrowSPairs benchmarks to evaluate biases in models like BERT and GPT-3.5, developed an automated Bias-Identification Framework with two-pronged approach for explicit/implicit bias detection, employed Bag-of-Words analysis, and applied fine-tuning with prompting and data augmentation.

Result: Fine-tuned models struggled with gender biases but excelled at racial bias identification. LLMs often over-relied on keywords. Enhancement strategies achieved up to 20% performance gains on implicit bias benchmarks, with fine-tuned models showing promising cross-dataset adaptability.

Conclusion: Despite some success, LLMs have limitations in bias detection and require targeted enhancement strategies. The proposed framework and fine-tuning approaches significantly improve implicit bias identification capabilities.

Abstract: Large Language Models (LLMs) inherit explicit and implicit biases from their training datasets. Identifying and mitigating biases in LLMs is crucial to ensure fair outputs, as they can perpetuate harmful stereotypes and misinformation. This study highlights the need to address biases in LLMs amid growing generative AI. We studied bias-specific benchmarks such as StereoSet and CrowSPairs to evaluate the existence of various biases in multiple generative models such as BERT and GPT 3.5. We proposed an automated Bias-Identification Framework to recognize various social biases in LLMs such as gender, race, profession, and religion. We adopted a two-pronged approach to detect explicit and implicit biases in text data. Results indicated fine-tuned models struggle with gender biases but excelled at identifying and avoiding racial biases. Our findings illustrated that despite having some success, LLMs often over-relied on keywords. To illuminate the capability of the analyzed LLMs in detecting implicit biases, we employed Bag-of-Words analysis and unveiled indications of implicit stereotyping within the vocabulary. To bolster the model performance, we applied an enhancement strategy involving fine-tuning models using prompting techniques and data augmentation of the bias benchmarks. The fine-tuned models exhibited promising adaptability during cross-dataset testing and significantly enhanced performance on implicit bias benchmarks, with performance gains of up to 20%.

Method: Three-stage approach: (1) efficiently identify minimal influenced neighborhoods via triangular structures, (2) apply sociological theories to quantify node importance for optimal privacy budget allocation, and (3) perform importance-weighted parameter updates to achieve certified modifications.

Result: Extensive experiments show CSGU outperforms existing methods, achieving superior performance in both utility preservation and unlearning effectiveness on SGNNs.

Conclusion: CSGU successfully addresses the limitations of existing unlearning methods for signed graphs by preserving sign information and sociological principles while providing certified privacy guarantees with minimal utility degradation.

Abstract: Signed graphs model complex relationships through positive and negative edges, with widespread real-world applications. Given the sensitive nature of such data, selective removal mechanisms have become essential for privacy protection. While graph unlearning enables the removal of specific data influences from Graph Neural Networks (GNNs), existing methods are designed for conventional GNNs and overlook the unique heterogeneous properties of signed graphs. When applied to Signed Graph Neural Networks (SGNNs), these methods lose critical sign information, degrading both model utility and unlearning effectiveness. To address these challenges, we propose Certified Signed Graph Unlearning (CSGU), which provides provable privacy guarantees while preserving the sociological principles underlying SGNNs. CSGU employs a three-stage method: (1) efficiently identifying minimal influenced neighborhoods via triangular structures, (2) applying sociological theories to quantify node importance for optimal privacy budget allocation, and (3) performing importance-weighted parameter updates to achieve certified modifications with minimal utility degradation. Extensive experiments demonstrate that CSGU outperforms existing methods, achieving superior performance in both utility preservation and unlearning effectiveness on SGNNs.

Method: Analyzes latent representations through Angular-Probabilistic Trajectory (APT) and introduces Jensen-Shannon Separability (JSS) metric to characterize hidden layer dynamics without text generation.

Result: Experiments on 40+ models and 20+ red teaming strategies show JSS metric has high consistency with safety rankings from red teaming, reproducing trends at <1% token and runtime cost.

Conclusion: N-GLARE provides an efficient, output-free evaluation proxy for real-time LLM safety diagnostics, enabling cost-effective and rapid safety assessment.

Abstract: Evaluating the safety robustness of LLMs is critical for their deployment. However, mainstream Red Teaming methods rely on online generation and black-box output analysis. These approaches are not only costly but also suffer from feedback latency, making them unsuitable for agile diagnostics after training a new model. To address this, we propose N-GLARE (A Non-Generative, Latent Representation-Efficient LLM Safety Evaluator). N-GLARE operates entirely on the model’s latent representations, bypassing the need for full text generation. It characterizes hidden layer dynamics by analyzing the APT (Angular-Probabilistic Trajectory) of latent representations and introducing the JSS (Jensen-Shannon Separability) metric. Experiments on over 40 models and 20 red teaming strategies demonstrate that the JSS metric exhibits high consistency with the safety rankings derived from Red Teaming. N-GLARE reproduces the discriminative trends of large-scale red-teaming tests at less than 1% of the token cost and the runtime cost, providing an efficient output-free evaluation proxy for real-time diagnostics.

Method: Unified hybrid Bayesian Deep Learning framework using variational inference for epistemic uncertainty and physics-informed stochastic perturbation scheme for aleatoric uncertainty, validated on ERA5 reanalysis dataset.

Result: Improved forecast accuracy, better-calibrated uncertainty quantification, and superior computational efficiency compared to state-of-the-art probabilistic diffusion models.

Conclusion: The framework successfully bridges Bayesian Deep Learning and ensemble prediction, providing a unified theoretical foundation for probabilistic weather forecasting with enhanced performance and efficiency.

Abstract: Weather forecasting is fundamentally challenged by the chaotic nature of the atmosphere, necessitating probabilistic approaches to quantify uncertainty. While traditional ensemble prediction (EPS) addresses this through computationally intensive simulations, recent advances in Bayesian Deep Learning (BDL) offer a promising but often disconnected alternative. We bridge these paradigms through a unified hybrid Bayesian Deep Learning framework for ensemble weather forecasting that explicitly decomposes predictive uncertainty into epistemic and aleatoric components, learned via variational inference and a physics-informed stochastic perturbation scheme modeling flow-dependent atmospheric dynamics, respectively. We further establish a unified theoretical framework that rigorously connects BDL and EPS, providing formal theorems that decompose total predictive uncertainty into epistemic and aleatoric components under the hybrid BDL framework. We validate our framework on the large-scale 40-year ERA5 reanalysis dataset (1979-2019) with 0.25° spatial resolution. Experimental results show that our method not only improves forecast accuracy and yields better-calibrated uncertainty quantification but also achieves superior computational efficiency compared to state-of-the-art probabilistic diffusion models. We commit to making our code open-source upon acceptance of this paper.

Method: Introduces Twice Sequential Monte Carlo Tree Search (TSMCTS) which addresses SMC’s limitations through variance reduction and mitigation of path degeneracy.

Result: TSMCTS outperforms both SMC baseline and modern MCTS across discrete and continuous environments, scaling better with sequential compute.

Conclusion: TSMCTS successfully combines SMC’s parallelization advantages with improved scaling through variance reduction and path degeneracy mitigation.

Abstract: Model-based reinforcement learning (RL) methods that leverage search are responsible for many milestone breakthroughs in RL. Sequential Monte Carlo (SMC) recently emerged as an alternative to the Monte Carlo Tree Search (MCTS) algorithm which drove these breakthroughs. SMC is easier to parallelize and more suitable to GPU acceleration. However, it also suffers from large variance and path degeneracy which prevent it from scaling well with increased search depth, i.e., increased sequential compute. To address these problems, we introduce Twice Sequential Monte Carlo Tree Search (TSMCTS). Across discrete and continuous environments TSMCTS outperforms the SMC baseline as well as a popular modern version of MCTS. Through variance reduction and mitigation of path degeneracy, TSMCTS scales favorably with sequential compute while retaining the properties that make SMC natural to parallelize.

Method: Uses EBind method with three complementary data sources: 6.7M automated multimodal quintuples, 1M human-annotated triples (negative/partial/positive matches), and 3.4M pre-existing captioned data. Trains a 1.8B-parameter image-text-video-audio-3D model.

Result: The 1.8B-parameter model outperforms models 4 to 17x larger in size. Introduces first high-quality consensus-annotated zero-shot classification benchmark between audio and point clouds.

Conclusion: EBind demonstrates that careful data curation and parameter-efficient methods can achieve superior performance compared to much larger models, with plans to open-source code, model weights, and datasets.

Abstract: We simplify space binding by focusing on two core components, a single encoder per modality and high-quality data; enabling training state-of-the-art models on a single GPU in a few hours as opposed to multiple days. We present EBind, an Easy, data-centric, and parameter-efficient method to Bind the embedding spaces of multiple contrastive models. We demonstrate that a simple 1.8B-parameter image-text-video-audio-3D model can outperform models 4 to 17x the size. The key to achieving this is a carefully curated dataset of three complementary data sources: i) 6.7M fully-automated multimodal quintuples sourced via SOTA retrieval models, ii) 1M diverse, semi-automated triples annotated by humans as negative, partial, or positive matches, and iii) 3.4M pre-existing captioned data items. We use 13 different evaluations to demonstrate the value of each data source. Due to limitations with existing benchmarks, we further introduce the first high-quality, consensus-annotated zero-shot classification benchmark between audio and PCs. In contrast to related work, we will open-source our code, model weights, and datasets.

Method: Uses object-centric Transformers to represent observations as object tokens plus action and reward tokens. Predicts token-level dynamics and interactions. Policy/value networks estimate token-level cause-effect relations for causality-guided decision-making.

Result: Consistently outperforms state-of-the-art agents in both sample efficiency and final performance on object-rich benchmarks.

Conclusion: Object-centric world modeling with causality-aware reinforcement learning provides superior performance in complex, multi-object environments compared to holistic approaches.

Abstract: World models have been developed to support sample-efficient deep reinforcement learning agents. However, it remains challenging for world models to accurately replicate environments that are high-dimensional, non-stationary, and composed of multiple objects with rich interactions since most world models learn holistic representations of all environmental components. By contrast, humans perceive the environment by decomposing it into discrete objects, facilitating efficient decision-making. Motivated by this insight, we propose \emph{Slot Transformer Imagination with CAusality-aware reinforcement learning} (STICA), a unified framework in which object-centric Transformers serve as the world model and causality-aware policy and value networks. STICA represents each observation as a set of object-centric tokens, together with tokens for the agent action and the resulting reward, enabling the world model to predict token-level dynamics and interactions. The policy and value networks then estimate token-level cause–effect relations and use them in the attention layers, yielding causality-guided decision-making. Experiments on object-rich benchmarks demonstrate that STICA consistently outperforms state-of-the-art agents in both sample efficiency and final performance.

Method: Transformer-based architecture with novel encoding scheme for efficient matrix and vector representation, enabling scalable inference with O(n²) memory complexity.

Result: Demonstrated effectiveness on application-driven linear problems including interpolation from spectral methods for boundary value problems and Newton method acceleration. Achieves competitive accuracy with significantly lower computational overhead.

Conclusion: General-purpose neural architectures like Algebraformer can effectively reduce complexity in traditional scientific computing pipelines for solving linear systems.

Abstract: Recent work in deep learning has opened new possibilities for solving classical algorithmic tasks using end-to-end learned models. In this work, we investigate the fundamental task of solving linear systems, particularly those that are ill-conditioned. Existing numerical methods for ill-conditioned systems often require careful parameter tuning, preconditioning, or domain-specific expertise to ensure accuracy and stability. In this work, we propose Algebraformer, a Transformer-based architecture that learns to solve linear systems end-to-end, even in the presence of severe ill-conditioning. Our model leverages a novel encoding scheme that enables efficient representation of matrix and vector inputs, with a memory complexity of $O(n^2)$, supporting scalable inference. We demonstrate its effectiveness on application-driven linear problems, including interpolation tasks from spectral methods for boundary value problems and acceleration of the Newton method. Algebraformer achieves competitive accuracy with significantly lower computational overhead at test time, demonstrating that general-purpose neural architectures can effectively reduce complexity in traditional scientific computing pipelines.

Method: Proposes a unified framework that classifies navigation intentions into sustained and transient categories, constructs sustained intention trees from historical trajectories, models dynamic transient intentions using Conditional Variational Autoencoder (CVAE), and uses non-local attention mechanism for global scenario consistency.

Result: Experiments on real AIS datasets demonstrate broad applicability across diverse scenarios with significant improvements in both ADE (Average Displacement Error) and FDE (Final Displacement Error) metrics.

Conclusion: The method improves explainability by explicitly revealing navigational intentions underlying each predicted trajectory while achieving superior prediction performance across various maritime scenarios.

Abstract: Vessel trajectory prediction is fundamental to intelligent maritime systems. Within this domain, short-term prediction of rapid behavioral changes in complex maritime environments has established multimodal trajectory prediction (MTP) as a promising research area. However, existing vessel MTP methods suffer from limited scenario applicability and insufficient explainability. To address these challenges, we propose a unified MTP framework incorporating explainable navigation intentions, which we classify into sustained and transient categories. Our method constructs sustained intention trees from historical trajectories and models dynamic transient intentions using a Conditional Variational Autoencoder (CVAE), while using a non-local attention mechanism to maintain global scenario consistency. Experiments on real Automatic Identification System (AIS) datasets demonstrates our method’s broad applicability across diverse scenarios, achieving significant improvements in both ADE and FDE. Furthermore, our method improves explainability by explicitly revealing the navigational intentions underlying each predicted trajectory.

Method: Systematic evaluation of task-agnostic representations from TabPFN and TabICL foundation models versus classical TableVectorizer feature engineering across outlier detection (ADBench) and supervised learning (TabArena Lite) tasks.

Result: TableVectorizer features achieve comparable or superior performance to foundation models while being up to three orders of magnitude (1000x) faster.

Conclusion: Classical feature engineering methods like TableVectorizer provide more efficient and effective task-agnostic representations than complex tabular foundation models for tabular data.

Abstract: Recent foundation models for tabular data achieve strong task-specific performance via in-context learning. Nevertheless, they focus on direct prediction by encapsulating both representation learning and task-specific inference inside a single, resource-intensive network. This work specifically focuses on representation learning, i.e., on transferable, task-agnostic embeddings. We systematically evaluate task-agnostic representations from tabular foundation models (TabPFN and TabICL) alongside with classical feature engineering (TableVectorizer) across a variety of application tasks as outlier detection (ADBench) and supervised learning (TabArena Lite). We find that simple TableVectorizer features achieve comparable or superior performance while being up to three orders of magnitude faster than tabular foundation models. The code is available at https://github.com/ContactSoftwareAI/TabEmbedBench.

Method: Introduces VAR that deliberately increases variance of model parameters during training, exploiting the finding that parameters with higher variance exhibit greater pruning robustness.

Result: Extensive empirical results demonstrate superior pruning robustness compared to existing methods, with theoretical analysis supporting convergence behavior.

Conclusion: VAR effectively mitigates adverse effects of pruning by promoting higher variance in weight distribution, providing pruning robustness without additional computational overhead.

Abstract: Deep neural networks achieve outstanding performance in visual recognition tasks, yet their large number of parameters makes them less practical for real-world applications. Recently, one-shot pruning has emerged as an effective strategy for reducing model size without additional training. However, models trained with standard objective functions often suffer a significant drop in accuracy after aggressive pruning. Some existing pruning-robust optimizers, such as SAM, and CrAM, mitigate this accuracy drop by guiding the model toward flatter regions of the parameter space, but they inevitably incur non-negligible additional computations. We propose a Variance Amplifying Regularizer (VAR) that deliberately increases the variance of model parameters during training. Our study reveals an intriguing finding that parameters with higher variance exhibit greater pruning robustness. VAR exploits this property by promoting such variance in the weight distribution, thereby mitigating the adverse effects of pruning. We further provide a theoretical analysis of its convergence behavior, supported by extensive empirical results demonstrating the superior pruning robustness of VAR.

Method: Uses a multi-scale VAE for physiologically-disentangled latent space, hierarchical text-to-biosignal pipeline with clinical metadata control, and interpretable diffusion process with Medical Attention module.

Result: Achieved state-of-the-art performance with Fréchet Audio Distance of 9.7, 92% attribute disentanglement score, 87.1% clinical validity, and improved rare disease classification accuracy by 11.3% when used for data augmentation.

Conclusion: H-LDM establishes a new direction for data augmentation in cardiac diagnostics, bridging data scarcity with interpretable clinical insights.

Abstract: Phonocardiogram (PCG) analysis is vital for cardiovascular disease diagnosis, yet the scarcity of labeled pathological data hinders the capability of AI systems. To bridge this, we introduce H-LDM, a Hierarchical Latent Diffusion Model for generating clinically accurate and controllable PCG signals from structured metadata. Our approach features: (1) a multi-scale VAE that learns a physiologically-disentangled latent space, separating rhythm, heart sounds, and murmurs; (2) a hierarchical text-to-biosignal pipeline that leverages rich clinical metadata for fine-grained control over 17 distinct conditions; and (3) an interpretable diffusion process guided by a novel Medical Attention module. Experiments on the PhysioNet CirCor dataset demonstrate state-of-the-art performance, achieving a Fréchet Audio Distance of 9.7, a 92% attribute disentanglement score, and 87.1% clinical validity confirmed by cardiologists. Augmenting diagnostic models with our synthetic data improves the accuracy of rare disease classification by 11.3%. H-LDM establishes a new direction for data augmentation in cardiac diagnostics, bridging data scarcity with interpretable clinical insights.

Method: Two complementary tools: 1) Intervention Efficiency (IE) - a capacity-aware metric that quantifies how efficiently a model identifies actionable true positives under limited intervention capacity, linking prediction to clinical utility. 2) Perturbation Validation Framework (PVF) - assesses model stability under data perturbations to identify models with invariant performance across noisy/shifted validation sets.

Result: Empirical evaluation on synthetic and real-world healthcare datasets shows that using IE and PVF enables selection of models that generalize more robustly and align with capacity constraints.

Conclusion: The proposed tools offer a new direction for addressing the Rashomon Effect in clinical settings by providing robust model assessment that considers both clinical utility and stability under data perturbations.

Abstract: In clinical machine learning, the coexistence of multiple models with comparable performance – a manifestation of the Rashomon Effect – poses fundamental challenges for trustworthy deployment and evaluation. Small, imbalanced, and noisy datasets, coupled with high-dimensional and weakly identified clinical features, amplify this multiplicity and make conventional validation schemes unreliable. As a result, selecting among equally performing models becomes uncertain, particularly when resource constraints and operational priorities are not considered by conventional metrics like F1 score. To address these issues, we propose two complementary tools for robust model assessment and selection: Intervention Efficiency (IE) and the Perturbation Validation Framework (PVF). IE is a capacity-aware metric that quantifies how efficiently a model identifies actionable true positives when only limited interventions are feasible, thereby linking predictive performance with clinical utility. PVF introduces a structured approach to assess the stability of models under data perturbations, identifying models whose performance remains most invariant across noisy or shifted validation sets. Empirical results on synthetic and real-world healthcare datasets show that using these tools facilitates the selection of models that generalize more robustly and align with capacity constraints, offering a new direction for tackling the Rashomon Effect in clinical settings.

Method: The authors derive a generalization theory for equality-constrained statistical learning problems and propose a practical algorithm that solves a sequence of unconstrained, empirical learning problems to approximate the constrained solution.

Result: The paper demonstrates the effectiveness of the proposed approach in three applications: fair learning, interpolating classifiers, and boundary value problems, showing that equality constraints enable new formulations and better handling of requirements like fairness.

Conclusion: The work provides theoretical foundations and practical algorithms for handling equality constraints in machine learning, overcoming limitations of traditional penalty methods and enabling more effective treatment of requirements involving parities and equalities.

Abstract: As machine learning applications grow increasingly ubiquitous and complex, they face an increasing set of requirements beyond accuracy. The prevalent approach to handle this challenge is to aggregate a weighted combination of requirement violation penalties into the training objective. To be effective, this approach requires careful tuning of these hyperparameters (weights), involving trial-and-error and cross-validation, which becomes ineffective even for a moderate number of requirements. These issues are exacerbated when the requirements involve parities or equalities, as is the case in fairness and boundary value problems. An alternative technique uses constrained optimization to formulate these learning problems. Yet, existing approximation and generalization guarantees do not apply to problems involving equality constraints. In this work, we derive a generalization theory for equality-constrained statistical learning problems, showing that their solutions can be approximated using samples and rich parametrizations. Using these results, we propose a practical algorithm based on solving a sequence of unconstrained, empirical learning problems. We showcase its effectiveness and the new formulations enabled by equality constraints in fair learning, interpolating classifiers, and boundary value problems.

Method: A simple, generalized irreversibility-regularized strategy that enforces hidden physical laws as soft constraints during PINN training.

Result: Reduces predictive errors by more than an order of magnitude across benchmarks including traveling waves, combustion, melting, corrosion, and crack propagation.

Conclusion: The framework is broadly applicable to PDE-governed physical systems and will significantly impact scientific machine learning.

Abstract: Physics-informed neural networks (PINNs) represent a new paradigm for solving partial differential equations (PDEs) by integrating physical laws into the learning process of neural networks. However, despite their foundational role, the hidden irreversibility implied by the Second Law of Thermodynamics is often neglected during training, leading to unphysical solutions or even training failures in conventional PINNs. In this paper, we identify this critical gap and introduce a simple, generalized, yet robust irreversibility-regularized strategy that enforces hidden physical laws as soft constraints during training. This approach ensures that the learned solutions consistently respect the intrinsic one-way nature of irreversible physical processes. Across a wide range of benchmarks spanning traveling wave propagation, steady combustion, ice melting, corrosion evolution, and crack propagation, we demonstrate that our regularization scheme reduces predictive errors by more than an order of magnitude, while requiring only minimal modification to existing PINN frameworks. We believe that the proposed framework is broadly applicable to a wide class of PDE-governed physical systems and will have significant impact within the scientific machine learning community.

Method: Conducted experiments analyzing the influence of LoRA rank on state-of-the-art backdoor attacks in FL, focusing on backdoor lifespan and persistence characteristics.

Result: Key finding: For optimally injected backdoors, lower LoRA ranks result in longer backdoor persistence after the attack ends. Also identified evaluation issues in current FL backdoor attack assessments.

Conclusion: The work contributes to more robust and fair evaluations of backdoor attacks in FL, enhancing reliability of risk assessments for critical FL systems.

Abstract: Large models adaptation through Federated Learning (FL) addresses a wide range of use cases and is enabled by Parameter-Efficient Fine-Tuning techniques such as Low-Rank Adaptation (LoRA). However, this distributed learning paradigm faces several security threats, particularly to its integrity, such as backdoor attacks that aim to inject malicious behavior during the local training steps of certain clients. We present the first analysis of the influence of LoRA on state-of-the-art backdoor attacks targeting model adaptation in FL. Specifically, we focus on backdoor lifespan, a critical characteristic in FL, that can vary depending on the attack scenario and the attacker’s ability to effectively inject the backdoor. A key finding in our experiments is that for an optimally injected backdoor, the backdoor persistence after the attack is longer when the LoRA’s rank is lower. Importantly, our work highlights evaluation issues of backdoor attacks against FL and contributes to the development of more robust and fair evaluations of backdoor attacks, enhancing the reliability of risk assessments for critical FL systems. Our code is publicly available.

Method: REST refines retrieval results to formulate query predictions and uses a QS-robust objective function for online adaptation. It employs gradient decoupling to prevent forgetting of general knowledge during adaptation.

Result: Extensive experiments on 20 benchmarks across three CMR tasks demonstrate the method’s effectiveness against query shift problems.

Conclusion: REST successfully addresses both online and diverse query shift challenges in cross-modal retrieval while preserving the model’s general knowledge and common space structure.

Abstract: Recently, the general-to-customized paradigm has emerged as the dominant approach for Cross-Modal Retrieval (CMR), which reconciles the distribution shift problem between the source domain and the target domain. However, existing general-to-customized CMR methods typically assume that the entire target-domain data is available, which is easily violated in real-world scenarios and thus inevitably suffer from the query shift (QS) problem. Specifically, query shift embraces the following two characteristics and thus poses new challenges to CMR. i) Online Shift: real-world queries always arrive in an online manner, rendering it impractical to access the entire query set beforehand for customization approaches; ii) Diverse Shift: even with domain customization, the CMR models struggle to satisfy queries from diverse users or scenarios, leaving an urgent need to accommodate diverse queries. In this paper, we observe that QS would not only undermine the well-structured common space inherited from the source model, but also steer the model toward forgetting the indispensable general knowledge for CMR. Inspired by the observations, we propose a novel method for achieving online and harmonious adaptation against QS, dubbed Robust adaptation with quEry ShifT (REST). To deal with online shift, REST first refines the retrieval results to formulate the query predictions and accordingly designs a QS-robust objective function on these predictions to preserve the well-established common space in an online manner. As for tackling the more challenging diverse shift, REST employs a gradient decoupling module to dexterously manipulate the gradients during the adaptation process, thus preventing the CMR model from forgetting the general knowledge. Extensive experiments on 20 benchmarks across three CMR tasks verify the effectiveness of our method against QS.

Method: FlowRoI estimates optical flow between consecutive frames to derive region of interest (RoI) masks covering migrating cells, then jointly encodes raw images and RoI masks using JPEG2000 for RoI-aware compression.

Result: FlowRoI achieves computational efficiency comparable to standard JPEG2000 (30 fps on Intel i7-1255U), higher PSNR in cellular regions, and 2.0-2.2x higher compression rates at matched PSNR.

Conclusion: FlowRoI provides an effective solution for high-throughput image compression in immune cell migration studies, enabling efficient data management while preserving critical cellular information.

Abstract: Autonomous migration is essential for the function of immune cells such as neutrophils and plays a pivotal role in diverse diseases. Recently, we introduced ComplexEye, a multi-lens array microscope comprising 16 independent aberration-corrected glass lenses arranged at the pitch of a 96-well plate, capable of capturing high-resolution movies of migrating cells. This architecture enables high-throughput live-cell video microscopy for migration analysis, supporting routine quantification of autonomous motility with strong potential for clinical translation. However, ComplexEye and similar high-throughput imaging platforms generate data at an exponential rate, imposing substantial burdens on storage and transmission. To address this challenge, we present FlowRoI, a fast optical-flow-based region of interest (RoI) extraction framework designed for high-throughput image compression in immune cell migration studies. FlowRoI estimates optical flow between consecutive frames and derives RoI masks that reliably cover nearly all migrating cells. The raw image and its corresponding RoI mask are then jointly encoded using JPEG2000 to enable RoI-aware compression. FlowRoI operates with high computational efficiency, achieving runtimes comparable to standard JPEG2000 and reaching an average throughput of about 30 frames per second on a modern laptop equipped with an Intel i7-1255U CPU. In terms of image quality, FlowRoI yields higher peak signal-to-noise ratio (PSNR) in cellular regions and achieves 2.0-2.2x higher compression rates at matched PSNR compared to standard JPEG2000.

Method: Mirage infusion technique enables diffusion models to alter atom states from existent to non-existent, creating MiAD - an equivariant joint diffusion model for de novo crystal generation.

Result: MiAD achieves 2.5x quality improvement over baseline models and reaches 8.2% S.U.N. rate on MP-20 dataset, substantially exceeding state-of-the-art approaches.

Conclusion: The mirage infusion technique successfully addresses the limitation of fixed atom numbers in diffusion models, enabling more flexible and higher-quality crystal material generation.

Abstract: In recent years, diffusion-based models have demonstrated exceptional performance in searching for simultaneously stable, unique, and novel (S.U.N.) crystalline materials. However, most of these models don’t have the ability to change the number of atoms in the crystal during the generation process, which limits the variability of model sampling trajectories. In this paper, we demonstrate the severity of this restriction and introduce a simple yet powerful technique, mirage infusion, which enables diffusion models to change the state of the atoms that make up the crystal from existent to non-existent (mirage) and vice versa. We show that this technique improves model quality by up to $\times2.5$ compared to the same model without this modification. The resulting model, Mirage Atom Diffusion (MiAD), is an equivariant joint diffusion model for de novo crystal generation that is capable of altering the number of atoms during the generation process. MiAD achieves an $8.2%$ S.U.N. rate on the MP-20 dataset, which substantially exceeds existing state-of-the-art approaches. The source code can be found at \href{https://github.com/andrey-okhotin/miad.git}{\texttt{github.com/andrey-okhotin/miad}}.

Method: Hybrid approach combining conditional variational autoencoder trained on paired PPG-APW data with conditional density estimator trained on labeled simulated APW segments, using hemodynamic simulations and unlabeled clinical data.

Result: The approach can detect fluctuations of cardiac output and stroke volume and outperforms supervised baseline in monitoring temporal changes of these biomarkers.

Conclusion: The proposed hybrid method successfully enables non-invasive estimation of key cardiovascular biomarkers from PPG signals, addressing the challenge of limited annotated data through simulation and unlabeled clinical data integration.

Abstract: Continuous cardiovascular monitoring can play a key role in precision health. However, some fundamental cardiac biomarkers of interest, including stroke volume and cardiac output, require invasive measurements, e.g., arterial pressure waveforms (APW). As a non-invasive alternative, photoplethysmography (PPG) measurements are routinely collected in hospital settings. Unfortunately, the prediction of key cardiac biomarkers from PPG instead of APW remains an open challenge, further complicated by the scarcity of annotated PPG measurements. As a solution, we propose a hybrid approach that uses hemodynamic simulations and unlabeled clinical data to estimate cardiovascular biomarkers directly from PPG signals. Our hybrid model combines a conditional variational autoencoder trained on paired PPG-APW data with a conditional density estimator of cardiac biomarkers trained on labeled simulated APW segments. As a key result, our experiments demonstrate that the proposed approach can detect fluctuations of cardiac output and stroke volume and outperform a supervised baseline in monitoring temporal changes in these biomarkers.

Method: Learns a stochastic map φ(x,u) such that φ(x,U) and Y|X=x follow approximately the same law, trained via Kullback-Leibler formulation without invertibility or adversarial training.

Result: Achieves performance competitive with or superior to state-of-the-art methods including kernel estimators, tree-based algorithms, and deep learning techniques, while remaining lightweight and easy to train.

Conclusion: CPFN provides an effective and efficient framework for conditional distribution estimation with theoretical consistency guarantees and practical advantages over existing methods.

Abstract: We introduce conditional push-forward neural networks (CPFN), a generative framework for conditional distribution estimation. Instead of directly modeling the conditional density $f_{Y|X}$, CPFN learns a stochastic map $\varphi=\varphi(x,u)$ such that $\varphi(x,U)$ and $Y|X=x$ follow approximately the same law, with $U$ a suitable random vector of pre-defined latent variables. This enables efficient conditional sampling and straightforward estimation of conditional statistics through Monte Carlo methods. The model is trained via an objective function derived from a Kullback-Leibler formulation, without requiring invertibility or adversarial training. We establish a near-asymptotic consistency result and demonstrate experimentally that CPFN can achieve performance competitive with, or even superior to, state-of-the-art methods, including kernel estimators, tree-based algorithms, and popular deep learning techniques, all while remaining lightweight and easy to train.

Method: Develop nnterp as a wrapper around NNsight with automatic module renaming and comprehensive validation testing, providing built-in implementations of common interpretability methods and direct access to attention probabilities.

Result: nnterp enables researchers to write intervention code once and deploy it across 50+ model variants spanning 16 architecture families, with built-in validation tests to verify compatibility with custom models.

Conclusion: nnterp bridges the gap between correctness and usability in mechanistic interpretability tooling by providing a unified interface while preserving original model implementations.

Abstract: Mechanistic interpretability research requires reliable tools for analyzing transformer internals across diverse architectures. Current approaches face a fundamental tradeoff: custom implementations like TransformerLens ensure consistent interfaces but require coding a manual adaptation for each architecture, introducing numerical mismatch with the original models, while direct HuggingFace access through NNsight preserves exact behavior but lacks standardization across models. To bridge this gap, we develop nnterp, a lightweight wrapper around NNsight that provides a unified interface for transformer analysis while preserving original HuggingFace implementations. Through automatic module renaming and comprehensive validation testing, nnterp enables researchers to write intervention code once and deploy it across 50+ model variants spanning 16 architecture families. The library includes built-in implementations of common interpretability methods (logit lens, patchscope, activation steering) and provides direct access to attention probabilities for models that support it. By packaging validation tests with the library, researchers can verify compatibility with custom models locally. nnterp bridges the gap between correctness and usability in mechanistic interpretability tooling.

Method: Develops theory starting from Hilbert space construction, then covers positive definite kernels, reproducing kernel Hilbert spaces (RKHS), and Hilbert-Schmidt operators, with emphasis on geometric foundations.

Result: Presents classical concepts (covariance, regression, information measures) through Hilbert space geometry, and introduces kernel density estimation, kernel embeddings, and Maximum Mean Discrepancy (MMD).

Conclusion: The exposition establishes fundamental geometric foundations for kernel methods, designed to support learning of advanced machine learning topics involving kernel-based approaches.

Abstract: These notes provide a self-contained introduction to kernel methods and their geometric foundations in machine learning. Starting from the construction of Hilbert spaces, we develop the theory of positive definite kernels, reproducing kernel Hilbert spaces (RKHS), and Hilbert-Schmidt operators, emphasizing their role in statistical estimation and representation of probability measures. Classical concepts such as covariance, regression, and information measures are revisited through the lens of Hilbert space geometry. We also introduce kernel density estimation, kernel embeddings of distributions, and the Maximum Mean Discrepancy (MMD). The exposition is designed to serve as a foundation for more advanced topics, including Gaussian processes, kernel Bayesian inference, and functional analytic approaches to modern machine learning.

Method: Uses perturbation-guided training to simulate disturbances, dual-path velocity field to capture trajectory deviations, and mixture-of-experts decoder with flow routing for dynamic capacity allocation.

Result: Extensive experiments show PAFM consistently outperforms strong baselines on both unconditional and conditional generation tasks.

Conclusion: PAFM effectively addresses temporal heterogeneity in time series generation by modeling perturbed trajectories, ensuring stable and structurally consistent generation.

Abstract: Time series generation is critical for a wide range of applications, which greatly supports downstream analytical and decision-making tasks. However, the inherent temporal heterogeneous induced by localized perturbations present significant challenges for generating structurally consistent time series. While flow matching provides a promising paradigm by modeling temporal dynamics through trajectory-level supervision, it fails to adequately capture abrupt transitions in perturbed time series, as the use of globally shared parameters constrains the velocity field to a unified representation. To address these limitations, we introduce \textbf{PAFM}, a \textbf{P}erturbation-\textbf{A}ware \textbf{F}low \textbf{M}atching framework that models perturbed trajectories to ensure stable and structurally consistent time series generation. The framework incorporates perturbation-guided training to simulate localized disturbances and leverages a dual-path velocity field to capture trajectory deviations under perturbation, enabling refined modeling of perturbed behavior to enhance the structural coherence. In order to further improve sensitivity to trajectory perturbations while enhancing expressiveness, a mixture-of-experts decoder with flow routing dynamically allocates modeling capacity in response to different trajectory dynamics. Extensive experiments on both unconditional and conditional generation tasks demonstrate that PAFM consistently outperforms strong baselines. Code is available at https://anonymous.4open.science/r/PAFM-03B2.

Method: CLO features: (1) coarse-grained head-wise approximate on-GPU caching with negligible management cost, (2) seamless combination of data prefetching and on-GPU persistent caching, (3) zero-copy transfer engine for full PCIe bandwidth utilization, and (4) GPU-centric synchronization to eliminate GPU stalls.

Result: Evaluation on two widely-used LLMs shows CLO achieves comparable accuracy to state-of-the-art systems while substantially minimizing CPU overhead, fully utilizing PCIe bandwidth, and improving decoding throughput by 9.3%-66.6%.

Conclusion: Algorithm-system co-design is essential for memory-constrained LLM inference on modern GPU platforms. CLO effectively addresses CPU bottlenecks in KVCache offloading systems.

Abstract: The growth of million-token LLMs exposes the scalability limits of inference systems, where the KVCache dominates memory usage and data transfer overhead. Recent offloading systems migrate the KVCache to CPU memory and incorporate top-k attention to reduce the volume of data transferred from the CPU, while further applying system-level optimizations such as on-GPU caching and prefetching to lower transfer overhead. However, they overlook the CPU bottleneck in three aspects: (1) substantial overhead of fine-grained dynamic cache management performed on the CPU side, (2) significant transfer overhead from poor PCIe bandwidth utilization caused by heavy gathering operations at the CPU side, and (3) GPU runtime bubbles introduced by coarse-grained CPU-centric synchronization. To address these challenges, we propose CLO, a CPU-light KVCache offloading system via algorithm-system co-design. CLO features: (1) a coarse-grained head-wise approximate on-GPU caching strategy with negligible cache management cost, (2) seamless combination of data prefetching and on-GPU persistent caching for lower transfer overhead, (3) a zero-copy transfer engine to fully exploit PCIe bandwidth, and a GPU-centric synchronization method to eliminate GPU stalls. Evaluation on two widely-used LLMs demonstrates that CLO achieves comparable accuracy to state-of-the-art systems, while substantially minimizing CPU overhead, fully utilizing PCIe bandwidth, thus improving decoding throughput by 9.3%-66.6%. Our results highlight that algorithm-system co-design is essential for memory-constrained LLM inference on modern GPU platforms. We open source CLO at https://github.com/CommediaJW/CLO.

Method: PepBFN directly models parameter distributions in continuous space, learns continuous distributions for discrete residue types, uses Gaussian mixture Bayesian flow for multimodal side chain states, and Matrix Fisher based Riemannian flow for residue orientations on SO(3) manifold.

Result: Experiments on side chain packing, reverse folding, and binder design tasks demonstrate PepBFN’s strong potential in computational peptide design, with smooth and coherent peptide generation.

Conclusion: PepBFN successfully addresses the limitations of current models by enabling joint continuous parameter modeling, capturing multimodal side chain states, and providing smooth Bayesian updates for improved peptide design performance.

Abstract: Diffusion and flow matching models have recently emerged as promising approaches for peptide binder design. Despite their progress, these models still face two major challenges. First, categorical sampling of discrete residue types collapses their continuous parameters into onehot assignments, while continuous variables (e.g., atom positions) evolve smoothly throughout the generation process. This mismatch disrupts the update dynamics and results in suboptimal performance. Second, current models assume unimodal distributions for side-chain torsion angles, which conflicts with the inherently multimodal nature of side chain rotameric states and limits prediction accuracy. To address these limitations, we introduce PepBFN, the first Bayesian flow network for full atom peptide design that directly models parameter distributions in fully continuous space. Specifically, PepBFN models discrete residue types by learning their continuous parameter distributions, enabling joint and smooth Bayesian updates with other continuous structural parameters. It further employs a novel Gaussian mixture based Bayesian flow to capture the multimodal side chain rotameric states and a Matrix Fisher based Riemannian flow to directly model residue orientations on the $\mathrm{SO}(3)$ manifold. Together, these parameter distributions are progressively refined via Bayesian updates, yielding smooth and coherent peptide generation. Experiments on side chain packing, reverse folding, and binder design tasks demonstrate the strong potential of PepBFN in computational peptide design.

Method: Separates features into two channels: continuous DDIM-based channel for numerical variables and discrete latent-path diffusion channel for categorical/discrete features, trained under unified conditional imputation objective.

Result: Achieves higher imputation accuracy, more stable sampling trajectories, and improved robustness across MCAR, MAR, and MNAR settings compared to existing methods.

Conclusion: Structure-aware diffusion processes are crucial for advancing deep learning approaches to incomplete tabular data with mixed attribute types.

Abstract: Incomplete data are common in real-world tabular applications, where numerical, categorical, and discrete attributes coexist within a single dataset. This heterogeneous structure presents significant challenges for existing diffusion-based imputation models, which typically assume a homogeneous feature space and rely on stochastic denoising trajectories. Such assumptions make it difficult to maintain conditional consistency, and they often lead to information collapse for categorical variables or instability when numerical variables require deterministic updates. These limitations indicate that a single diffusion process is insufficient for mixed-type tabular imputation. We propose a hybrid deterministic diffusion framework that separates heterogeneous features into two complementary generative channels. A continuous DDIM-based channel provides efficient and stable deterministic denoising for numerical variables, while a discrete latent-path diffusion channel, inspired by loopholing-based discrete diffusion, models categorical and discrete features without leaving their valid sample manifolds. The two channels are trained under a unified conditional imputation objective, enabling coherent reconstruction of mixed-type incomplete data. Extensive experiments on multiple real-world datasets show that the proposed framework achieves higher imputation accuracy, more stable sampling trajectories, and improved robustness across MCAR, MAR, and MNAR settings compared with existing diffusion-based and classical methods. These results demonstrate the importance of structure-aware diffusion processes for advancing deep learning approaches to incomplete tabular data.

Method: Developed Warping Index based on the assumption that correct preservation of empty regions between points is crucial for faithful visual representation.

Result: Proposed a new metric that specifically addresses visual distortion issues in dimensionality reduction projections.

Conclusion: Warping Index provides a valuable tool for quantifying projection quality with emphasis on visual faithfulness and empty region preservation.

Abstract: Dimensionality Reduction (DR) techniques are commonly used for the visual exploration and analysis of high-dimensional data due to their ability to project datasets of high-dimensional points onto the 2D plane. However, projecting datasets in lower dimensions often entails some distortion, which is not necessarily easy to recognize but can lead users to misleading conclusions. Several Projection Quality Metrics (PQMs) have been developed as tools to quantify the goodness-of-fit of a DR projection; however, they mostly focus on measuring how well the projection captures the global or local structure of the data, without taking into account the visual distortion of the resulting plots, thus often ignoring the presence of outliers or artifacts that can mislead a visual analysis of the projection. In this work, we introduce the Warping Index (WI), a new metric for measuring the quality of DR projections onto the 2D plane, based on the assumption that the correct preservation of empty regions between points is of crucial importance towards a faithful visual representation of the data.

Method: Applied task arithmetic to closed-vocabulary image classification models with different pre-training schemes, analyzed weight disentanglement properties, and compared with linear probing as a baseline.

Result: Found that weight disentanglement enabling task arithmetic is a general consequence of pre-training, works well with closed-vocabulary vision transformers, and achieves high task addition performance comparable to linear probing.

Conclusion: Task arithmetic can be successfully applied to closed-vocabulary models, expanding its applicability and enabling more efficient multi-task deployment of pre-trained models.

Abstract: Task arithmetic has recently emerged as a promising method for editing pre-trained \textit{open-vocabulary} models, offering a cost-effective alternative to standard multi-task fine-tuning. However, despite the abundance of \textit{closed-vocabulary} models that are not pre-trained with language supervision, applying task arithmetic to these models remains unexplored. In this paper, we deploy and study task addition in closed-vocabulary image classification models. We consider different pre-training schemes and find that \textit{weight disentanglement} – the property enabling task arithmetic – is a general consequence of pre-training, as it appears in different pre-trained closed-vocabulary models. In fact, we find that pre-trained closed-vocabulary vision transformers can also be edited with task arithmetic, achieving high task addition performance and enabling the efficient deployment of multi-task models. Finally, we demonstrate that simple linear probing is a competitive baseline to task addition. Overall, our findings expand the applicability of task arithmetic to a broader class of pre-trained models and open the way for more efficient use of pre-trained models in diverse settings.

Method: Combines three mechanisms: LLM-based hierarchical TODO decomposition for planning, history-aware causal reflection for failure analysis, and gradient-based optimization for systematic improvement.

Result: Achieves 67% zero-shot success rate on ALFWorld benchmark tasks on first trial without prior experience or demonstrations, with 67-78% improvement in cross-task transfer.

Conclusion: Synergistic integration of complementary learning mechanisms enables robust zero-shot generalization approaching few-shot baselines.

Abstract: Enabling agents to learn from experience and generalize across diverse tasks without task-specific training remains a fundamental challenge in reinforcement learning and decision-making. While recent approaches have explored episodic memory (Reflexion), gradient-based prompt optimization (TextGrad),and hierarchical task decomposition independently, their potential for synergistic integration remains unexplored. We introduce ReflexGrad, a novel architecture that tightly couples three complementary mechanisms: (1) LLM-based hierarchical TODO decomposition for strategic planning, (2) history-aware causal reflection that analyzes recent action patterns to identify failure root causes and enable within-trial learning, and (3) gradient-based optimization for systematic improvement. Unlike prior work relying on few-shot demonstrations, our system achieves true zero-shot generalization through pure LLM semantic reasoning,requiring no task-specific examples, fine-tuning, or hardcoded similarity metrics. Evaluated on ALFWorld benchmark tasks, ReflexGrad demonstrates 67% zero-shot success rate on Trial 0 without any prior task experience or demonstrations, establishing effective performance on first exposure. Through empirical analysis, we identify the architectural mechanisms underlying stable convergence (zero action loops) and effective cross-task transfer (67% to 78% improvement).Our work demonstrates that synergistic integration of complementary learning mechanisms enables robust zero-shot generalization that approaches few-shot baselines from prior work.

Method: Integrates expert knowledge through fuzzy pseudo-counts into EM framework, reformulating as Maximum A Posteriori estimation to guide learning with limited data.

Result: Outperforms standard EM in synthetic medical simulations under low-data and high-noise conditions; successfully recovers clinically coherent POMDP in Myasthenia Gravis case study.

Conclusion: Fuzzy MAP EM is a practical tool for data-efficient modeling in healthcare, effectively leveraging expert knowledge when data is limited.

Abstract: Learning the parameters of Partially Observable Markov Decision Processes (POMDPs) from limited data is a significant challenge. We introduce the Fuzzy MAP EM algorithm, a novel approach that incorporates expert knowledge into the parameter estimation process by enriching the Expectation Maximization (EM) framework with fuzzy pseudo-counts derived from an expert-defined fuzzy model. This integration naturally reformulates the problem as a Maximum A Posteriori (MAP) estimation, effectively guiding learning in environments with limited data. In synthetic medical simulations, our method consistently outperforms the standard EM algorithm under both low-data and high-noise conditions. Furthermore, a case study on Myasthenia Gravis illustrates the ability of the Fuzzy MAP EM algorithm to recover a clinically coherent POMDP, demonstrating its potential as a practical tool for data-efficient modeling in healthcare.

Method: Systematic experiments requesting LLMs to produce outputs following simple probabilistic distributions, testing binary outputs with specific probability targets.

Result: All modern LLMs tested grossly fail to follow distributions - requesting “1” 49% of the time produces “0” nearly 100% of the time, showing step function-like behavior.

Conclusion: LLMs exhibit deterministic behavior by exclusively generating the output with marginally highest probability, overriding even strong in-built biases, which hinders probabilistic exploration needed for scientific creativity.

Abstract: Scientific idea generation and selection requires exploration following a target probability distribution. In contrast, current AI benchmarks have objectively correct answers, and training large language models (LLMs) via reinforcement learning against these benchmarks discourages probabilistic exploration. Here, we conducted systematic experiments requesting LLMs to produce outputs following simple probabilistic distributions, and found that all modern LLMs tested grossly fail to follow the distributions. For example, requesting a binary output of “1” 49% of the time produces an answer of “0” nearly 100% of the time. This step function-like behavior of near-exclusively generating the output with marginally highest probability even overrules even strong in-built LLM biases.

Method: Adapformer uses a dual-stage encoder-decoder architecture with Adaptive Channel Enhancer (ACE) for enriching embeddings by selectively incorporating dependencies, and Adaptive Channel Forecaster (ACF) for refining predictions by focusing on relevant covariates while reducing noise.

Result: Rigorous testing on diverse datasets shows Adapformer achieves superior performance over existing models, enhancing both predictive accuracy and computational efficiency.

Conclusion: Adapformer represents state-of-the-art in multivariate time series forecasting by effectively addressing the limitations of both CI and CD approaches through adaptive channel management.

Abstract: In multivariate time series forecasting (MTSF), accurately modeling the intricate dependencies among multiple variables remains a significant challenge due to the inherent limitations of traditional approaches. Most existing models adopt either \textbf{channel-independent} (CI) or \textbf{channel-dependent} (CD) strategies, each presenting distinct drawbacks. CI methods fail to leverage the potential insights from inter-channel interactions, resulting in models that may not fully exploit the underlying statistical dependencies present in the data. Conversely, CD approaches often incorporate too much extraneous information, risking model overfitting and predictive inefficiency. To address these issues, we introduce the Adaptive Forecasting Transformer (\textbf{Adapformer}), an advanced Transformer-based framework that merges the benefits of CI and CD methodologies through effective channel management. The core of Adapformer lies in its dual-stage encoder-decoder architecture, which includes the \textbf{A}daptive \textbf{C}hannel \textbf{E}nhancer (\textbf{ACE}) for enriching embedding processes and the \textbf{A}daptive \textbf{C}hannel \textbf{F}orecaster (\textbf{ACF}) for refining the predictions. ACE enhances token representations by selectively incorporating essential dependencies, while ACF streamlines the decoding process by focusing on the most relevant covariates, substantially reducing noise and redundancy. Our rigorous testing on diverse datasets shows that Adapformer achieves superior performance over existing models, enhancing both predictive accuracy and computational efficiency, thus making it state-of-the-art in MTSF.

Method: Analyzed health screening data using Random Forest, XGBoost, and LightGBM algorithms with cross-sectional design to classify current smoking status based on biomarkers including body measurements, blood tests, and demographics.

Result: Random Forest performed best with AUC of 0.926, reliably distinguishing high-risk individuals. SHAP analysis identified blood pressure, triglyceride levels, liver enzymes, and kidney function indicators as strongest predictors.

Conclusion: Machine learning models can effectively identify smoking-related health risks using routine health screening data, with key biomarkers providing interpretable signals for early detection and intervention.

Abstract: Smoking continues to be a major preventable cause of death worldwide, affecting millions through damage to the heart, metabolism, liver, and kidneys. However, current medical screening methods often miss the early warning signs of smoking-related health problems, leading to late-stage diagnoses when treatment options become limited. This study presents a systematic comparative evaluation of machine learning approaches for smoking-related health risk assessment, emphasizing clinical interpretability and practical deployment over algorithmic innovation. We analyzed health screening data from 55,691 individuals, examining various health indicators, including body measurements, blood tests, and demographic information. We tested three advanced prediction algorithms - Random Forest, XGBoost, and LightGBM - to determine which could most accurately identify people at high risk. This study employed a cross-sectional design to classify current smoking status based on health screening biomarkers, not to predict future disease development. Our Random Forest model performed best, achieving an Area Under the Curve (AUC) of 0.926, meaning it could reliably distinguish between high-risk and lower-risk individuals. Using SHAP (SHapley Additive exPlanations) analysis to understand what the model was detecting, we found that key health markers played crucial roles in prediction: blood pressure levels, triglyceride concentrations, liver enzyme readings, and kidney function indicators (serum creatinine) were the strongest signals of declining health in smokers.

Method: Substitute L2 penalty with decoupled smooth Huber regularizer that decays parameters quadratically below threshold δ and linearly above δ, integrated with Adam-family optimizers at O(1) extra cost.

Result: 10-15% faster convergence, 4-point perplexity reduction, 2.5-4.7% downstream task improvement, 20-30% memory savings after pruning, and visibly sparser weights without tuning decay coefficients.

Conclusion: AdamHuberDecay provides a simple, principled approach for more efficient and resilient training of next-generation foundational generative transformers.

Abstract: Adaptive optimizers with decoupled weight decay, such as AdamW, are the de facto standard for pre-training large transformer-based generative models. Yet the quadratic nature of the $\ell_2$ penalty embedded in weight decay drives all parameters toward the origin at the same rate, making the update vulnerable to rare but extreme gradient directions and often over-penalizing well-conditioned coordinates. We propose AdamHuberDecay, a drop-in replacement for AdamW that substitutes the $\ell_2$ penalty with a decoupled smooth Huber regularizer. The resulting update decays parameters quadratically while their magnitude remains below a threshold $δ$, and linearly ($\ell_1$-like) once they exceed $δ$, yielding (i) bounded regularization gradients, (ii) invariance to per-coordinate second-moment rescaling, and (iii) stronger sparsity pressure on overgrown weights. We derive the closed-form decoupled Huber decay step and show how to integrate it with any Adam-family optimizer at $O(1)$ extra cost. Extensive experiments on GPT-2 and GPT-3 pre-training demonstrate that AdamHuberDecay (a) converges 10-15% faster in wall-clock time, (b) reduces validation perplexity by up to 4 points, (c) delivers performance improvements of 2.5-4.7% across downstream tasks, and (d) yields visibly sparser weight histograms that translate into 20-30% memory savings after magnitude pruning, without tuning the decay coefficient beyond the default grid used for AdamW. Ablations confirm robustness to outlier gradients and large-batch regimes, together with theoretical analyses that bound the expected parameter norm under noisy updates. AdamHuberDecay therefore provides a simple, principled path toward more efficient and resilient training of next-generation foundational generative transformers.

Method: LAUD combines LLMs with active learning, starting with zero-shot learning to create an initial label set, then iteratively improving through active learning cycles.

Result: Experimental results show LAUD-derived LLMs outperform zero-shot or few-shot learning approaches on commodity name classification tasks.

Conclusion: LAUD effectively addresses the labeled data scarcity problem by integrating LLMs with active learning, demonstrating practical utility in real-world applications.

Abstract: Large language models (LLMs) have shown a remarkable ability to generalize beyond their pre-training data, and fine-tuning LLMs can elevate performance to human-level and beyond. However, in real-world scenarios, lacking labeled data often prevents practitioners from obtaining well-performing models, thereby forcing practitioners to highly rely on prompt-based approaches that are often tedious, inefficient, and driven by trial and error. To alleviate this issue of lacking labeled data, we present a learning framework integrating LLMs with active learning for unlabeled dataset (LAUD). LAUD mitigates the cold-start problem by constructing an initial label set with zero-shot learning. Experimental results show that LLMs derived from LAUD outperform LLMs with zero-shot or few-shot learning on commodity name classification tasks, demonstrating the effectiveness of LAUD.

Method: Developed a tailored Gaussian process (GP) model that captures rating dynamics over time while accounting for review heterogeneity. Tested on 121,123 Yelp ratings comparing predictive power against traditional methods.

Result: GP model significantly more accurate, reducing mean absolute error by 10.2% compared to sample mean in predicting future ratings.

Conclusion: Moving beyond simple means to GP-based aggregation provides more informative and adaptive rating scores that better signal expected customer satisfaction, with important implications for online reputation systems.

Abstract: Online ratings influence customer decision-making, yet standard aggregation methods, such as the sample mean, fail to adapt to quality changes over time and ignore review heterogeneity (e.g., review sentiment, a review’s helpfulness). To address these challenges, we demonstrate the value of using the Gaussian process (GP) framework for rating aggregation. Specifically, we present a tailored GP model that captures the dynamics of ratings over time while additionally accounting for review heterogeneity. Based on 121,123 ratings from Yelp, we compare the predictive power of different rating aggregation methods in predicting future ratings, thereby finding that the GP model is considerably more accurate and reduces the mean absolute error by 10.2% compared to the sample mean. Our findings have important implications for marketing practitioners and customers. By moving beyond means, designers of online reputation systems can display more informative and adaptive aggregated rating scores that are accurate signals of expected customer satisfaction.

Method: Created a reproducible leaderboard hosted on Hugging Face using the original Tox21 Challenge dataset, with standardized baseline methods and API calls for inference.

Result: The original 2015 DeepTox ensemble method and 2017 descriptor-based self-normalizing neural networks still rank among top performers, suggesting limited progress in toxicity prediction over the past decade.

Conclusion: Substantial progress in toxicity prediction may not have been achieved as previously assumed, highlighting the need for standardized benchmarks and reproducible evaluation frameworks.

Abstract: Deep learning’s rise since the early 2010s has transformed fields like computer vision and natural language processing and strongly influenced biomedical research. For drug discovery specifically, a key inflection - akin to vision’s “ImageNet moment” - arrived in 2015, when deep neural networks surpassed traditional approaches on the Tox21 Data Challenge. This milestone accelerated the adoption of deep learning across the pharmaceutical industry, and today most major companies have integrated these methods into their research pipelines. After the Tox21 Challenge concluded, its dataset was included in several established benchmarks, such as MoleculeNet and the Open Graph Benchmark. However, during these integrations, the dataset was altered and labels were imputed or manufactured, resulting in a loss of comparability across studies. Consequently, the extent to which bioactivity and toxicity prediction methods have improved over the past decade remains unclear. To this end, we introduce a reproducible leaderboard, hosted on Hugging Face with the original Tox21 Challenge dataset, together with a set of baseline and representative methods. The current version of the leaderboard indicates that the original Tox21 winner - the ensemble-based DeepTox method - and the descriptor-based self-normalizing neural networks introduced in 2017, continue to perform competitively and rank among the top methods for toxicity prediction, leaving it unclear whether substantial progress in toxicity prediction has been achieved over the past decade. As part of this work, we make all baselines and evaluated models publicly accessible for inference via standardized API calls to Hugging Face Spaces.

Method: The paper formalizes level-k thinking (from behavioral economics) where users try to outsmart peers by looking one step ahead, and contrasts this with collective reasoning where users coordinate actions by optimizing their joint impact on the model.

Result: Individual level-k thinking accelerates convergence to equilibrium but doesn’t change the equilibrium itself, providing no long-term benefit for individuals. Collective coordination reveals benefits and limits of coordination, with alignment between learner’s and users’ utilities emerging as a key concept.

Conclusion: The analysis connects strategic classification, performative prediction, and algorithmic collective action, highlighting how user coordination fundamentally changes the dynamics compared to individual strategic behavior, with utility alignment becoming crucial for understanding platform-user interactions.

Abstract: On many learning platforms, the optimization criteria guiding model training reflect the priorities of the designer rather than those of the individuals they affect. Consequently, users may act strategically to obtain more favorable outcomes, effectively contesting the platform’s predictions. While past work has studied strategic user behavior on learning platforms, the focus has largely been on strategic responses to a deployed model, without considering the behavior of other users. In contrast, look-ahead reasoning takes into account that user actions are coupled, and – at scale – impact future predictions. Within this framework, we first formalize level-$k$ thinking, a concept from behavioral economics, where users aim to outsmart their peers by looking one step ahead. We show that, while convergence to an equilibrium is accelerated, the equilibrium remains the same, providing no benefit of higher-level reasoning for individuals in the long run. Then, we focus on collective reasoning, where users take coordinated actions by optimizing through their joint impact on the model. By contrasting collective with selfish behavior, we characterize the benefits and limits of coordination; a new notion of alignment between the learner’s and the users’ utilities emerges as a key concept. We discuss connections to several related mathematical frameworks, including strategic classification, performative prediction, and algorithmic collective action.

Method: Developed SparseST framework that pioneers exploiting data sparsity for efficient spatiotemporal modeling. Designed a multi-objective composite loss function to explore and approximate the Pareto front between model performance and computational efficiency.

Result: The framework provides a practical guide for practitioners to adjust models according to computational resource constraints and performance requirements of downstream tasks.

Conclusion: SparseST successfully addresses the computational inefficiency of spatiotemporal models by focusing on data redundancy rather than model capacity reduction, enabling deployment on resource-constrained edge devices while preserving performance through multi-objective optimization.

Abstract: Spatiotemporal data mining (STDM) has a wide range of applications in various complex physical systems (CPS), i.e., transportation, manufacturing, healthcare, etc. Among all the proposed methods, the Convolutional Long Short-Term Memory (ConvLSTM) has proved to be generalizable and extendable in different applications and has multiple variants achieving state-of-the-art performance in various STDM applications. However, ConvLSTM and its variants are computationally expensive, which makes them inapplicable in edge devices with limited computational resources. With the emerging need for edge computing in CPS, efficient AI is essential to reduce the computational cost while preserving the model performance. Common methods of efficient AI are developed to reduce redundancy in model capacity (i.e., model pruning, compression, etc.). However, spatiotemporal data mining naturally requires extensive model capacity, as the embedded dependencies in spatiotemporal data are complex and hard to capture, which limits the model redundancy. Instead, there is a fairly high level of data and feature redundancy that introduces an unnecessary computational burden, which has been largely overlooked in existing research. Therefore, we developed a novel framework SparseST, that pioneered in exploiting data sparsity to develop an efficient spatiotemporal model. In addition, we explore and approximate the Pareto front between model performance and computational efficiency by designing a multi-objective composite loss function, which provides a practical guide for practitioners to adjust the model according to computational resource constraints and the performance requirements of downstream tasks.

Method: RECAP (RL with Experience and Corrections via Advantage-conditioned Policies) uses advantage conditioning for RL training, starting with offline RL pre-training of a generalist VLA model, then specializing it through on-robot data collection that includes demonstrations, on-policy data, and expert teleoperated interventions.

Result: The method achieves successful performance on real-world tasks: folding laundry in homes, assembling boxes, and making espresso drinks. On hardest tasks, RECAP more than doubles task throughput and roughly halves failure rates.

Conclusion: RECAP provides an effective framework for improving VLA models through real-world deployment, demonstrating significant performance gains on complex manipulation tasks by leveraging heterogeneous data sources during the reinforcement learning process.

Abstract: We study how vision-language-action (VLA) models can improve through real-world deployments via reinforcement learning (RL). We present a general-purpose method, RL with Experience and Corrections via Advantage-conditioned Policies (RECAP), that provides for RL training of VLAs via advantage conditioning. Our method incorporates heterogeneous data into the self-improvement process, including demonstrations, data from on-policy collection, and expert teleoperated interventions provided during autonomous execution. RECAP starts by pre-training a generalist VLA with offline RL, which we call $π^{}_{0.6}$, that can then be specialized to attain high performance on downstream tasks through on-robot data collection. We show that the $π^{}_{0.6}$ model trained with the full RECAP method can fold laundry in real homes, reliably assemble boxes, and make espresso drinks using a professional espresso machine. On some of the hardest tasks, RECAP more than doubles task throughput and roughly halves the task failure rate.

Method: Extract features via follow-up prompts and response probabilities, then train linear models on these representations.

Result: Produces reliable predictors of model performance, often outperforms white-box linear predictors, and detects adversarial influences and model misrepresentation.

Conclusion: Black-box feature extraction via follow-up prompts enables effective prediction of LLM behavior and detection of model states without internal access.

Abstract: As large language models (LLMs) are increasingly relied on in AI systems, predicting when they make mistakes is crucial. While a great deal of work in the field uses internal representations to interpret model behavior, these representations are inaccessible when given solely black-box access through an API. In this paper, we extract features of LLMs in a black-box manner by using follow-up prompts and taking the probabilities of different responses as representations to train reliable predictors of model behavior. We demonstrate that training a linear model on these low-dimensional representations produces reliable and generalizable predictors of model performance at the instance level (e.g., if a particular generation correctly answers a question). Remarkably, these can often outperform white-box linear predictors that operate over a model’s hidden state or the full distribution over its vocabulary. In addition, we demonstrate that these extracted features can be used to evaluate more nuanced aspects of a language model’s state. For instance, they can be used to distinguish between a clean version of GPT-4o-mini and a version that has been influenced via an adversarial system prompt that answers question-answering tasks incorrectly or introduces bugs into generated code. Furthermore, they can reliably distinguish between different model architectures and sizes, enabling the detection of misrepresented models provided through an API (e.g., identifying if GPT-3.5 is supplied instead of GPT-4o-mini).

Method: IPAD consists of a Prompt Inverter that identifies predicted prompts that could have generated the input text, and two Distinguishers that examine the probability that input texts align with the predicted prompts.

Result: IPAD outperforms strongest baselines by 9.05% (Average Recall) on in-distribution data, 12.93% (AUROC) on out-of-distribution data, and 5.48% (AUROC) on attacked data, while also performing robustly on structured datasets.

Conclusion: IPAD enhances AI detection trustworthiness by providing interpretable evidence through direct examination of decision-making processes, supporting its state-of-the-art detection performance.

Abstract: Large Language Models (LLMs) have attained human-level fluency in text generation, which complicates the distinguishing between human-written and LLM-generated texts. This increases the risk of misuse and highlights the need for reliable detectors. Yet, existing detectors exhibit poor robustness on out-of-distribution (OOD) data and attacked data, which is critical for real-world scenarios. Also, they struggle to provide interpretable evidence to support their decisions, thus undermining the reliability. In light of these challenges, we propose IPAD (Inverse Prompt for AI Detection), a novel framework consisting of a Prompt Inverter that identifies predicted prompts that could have generated the input text, and two Distinguishers that examine the probability that the input texts align with the predicted prompts. Empirical evaluations demonstrate that IPAD outperforms the strongest baselines by 9.05% (Average Recall) on in-distribution data, 12.93% (AUROC) on out-of-distribution data, and 5.48% (AUROC) on attacked data. IPAD also performs robustly on structured datasets. Furthermore, an interpretability assessment is conducted to illustrate that IPAD enhances the AI detection trustworthiness by allowing users to directly examine the decision-making evidence, which provides interpretable support for its state-of-the-art detection results.

Method: Proposes a probabilistic framework that formalizes optimal inference-time scaling under i.i.d. assumptions, derives theoretical lower bounds on required samples, and develops OptScale algorithm with LM-based predictor to estimate parameters and dynamically determine optimal sample count.

Result: Extensive experiments on reasoning benchmarks (MATH-500, GSM8K, AIME, AMC) show OptScale significantly reduces sampling overhead while matching or exceeding state-of-the-art reasoning performance.

Conclusion: Provides both theoretical foundation and practical solution for principled inference-time scaling, addressing efficient deployment of LLMs for complex reasoning tasks.

Abstract: Inference-time scaling has emerged as a powerful technique for enhancing the reasoning performance of Large Language Models (LLMs). However, existing approaches often rely on heuristic strategies for parallel sampling, lacking a principled foundation. To address this gap, we propose a probabilistic framework that formalizes the optimality of inference-time scaling under the assumption that parallel samples are independently and identically distributed (i.i.d.), and where the Best-of-N selection strategy follows a probability distribution that can be estimated. Within this framework, we derive a theoretical lower bound on the required number of samples to achieve a target performance level, providing the first principled guidance for compute-efficient scaling. Leveraging this insight, we develop \textsc{OptScale}, a practical algorithm that dynamically determines the optimal number of sampled responses. \textsc{OptScale} employs a language model-based predictor to estimate probabilistic prior parameters, enabling the decision of the minimal number of samples needed that satisfy predefined performance thresholds and confidence levels. Extensive experiments on representative reasoning benchmarks (including MATH-500, GSM8K, AIME, and AMC) demonstrate that \textsc{OptScale} significantly reduces sampling overhead while remaining better or on par with state-of-the-art reasoning performance. Our work offers both a theoretical foundation and a practical solution for principled inference-time scaling, addressing a critical gap in the efficient deployment of LLMs for complex reasoning. The source code is publicly available at https://github.com/Albertwyk/OptScale.

Method: Uses a masked autoencoder (MAE) with likelihood-based approach for heterogeneous data types, transformer architecture to capture dependencies, and novel training scheme leveraging samples with diverse missingness patterns.

Result: VHGM-MAE outperforms existing methods in both missing value imputation and synthetic data generation tasks.

Conclusion: The proposed VHGM-MAE successfully addresses key technical challenges in healthcare data modeling and demonstrates superior performance compared to existing approaches.

Abstract: Virtual Human Generative Model (VHGM) is a generative model that approximates the joint probability over more than 2000 human healthcare-related attributes. This paper presents the core algorithm, VHGM-MAE, a masked autoencoder (MAE) tailored for handling high-dimensional, sparse healthcare data. VHGM-MAE tackles four key technical challenges: (1) heterogeneity of healthcare data types, (2) probability distribution modeling, (3) systematic missingness in the training dataset arising from multiple data sources, and (4) the high-dimensional, small-$n$-large-$p$ problem. To address these challenges, VHGM-MAE employs a likelihood-based approach to model distributions with heterogeneous types, a transformer-based MAE to capture complex dependencies among observed and missing attributes, and a novel training scheme that effectively leverages available samples with diverse missingness patterns to mitigate the small-n-large-p problem. Experimental results demonstrate that VHGM-MAE outperforms existing methods in both missing value imputation and synthetic data generation.

Method: In each round, FedEntOpt selects clients to maximize the entropy of the aggregated label distribution, ensuring the global model is exposed to data from all available classes.

Result: Outperforms state-of-the-art algorithms by up to 6% in classification accuracy, achieves over 30% gains in low participation scenarios, and can enhance existing algorithms’ accuracy by more than 40%. Performance unaffected by differential privacy.

Conclusion: FedEntOpt effectively mitigates label skew in federated learning while maintaining privacy and can be flexibly combined with existing methods for significant performance improvements.

Abstract: Although deep learning has revolutionized domains such as natural language processing and computer vision, its dependence on centralized datasets raises serious privacy concerns. Federated learning addresses this issue by enabling multiple clients to collaboratively train a global deep learning model without compromising their data privacy. However, the performance of such a model degrades under label skew, where the label distribution differs between clients. To overcome this issue, a novel method called FedEntOpt is proposed. In each round, it selects clients to maximize the entropy of the aggregated label distribution, ensuring that the global model is exposed to data from all available classes. Extensive experiments on multiple benchmark datasets show that the proposed method outperforms several state-of-the-art algorithms by up to 6% in classification accuracy under standard settings regardless of the model size, while achieving gains of over 30% in scenarios with low participation rates and client dropout. In addition, FedEntOpt offers the flexibility to be combined with existing algorithms, enhancing their classification accuracy by more than 40%. Importantly, its performance remains unaffected even when differential privacy is applied.

Method: Proposes Higher-Order Transformer (HOT) with factorized attention represented as sums of Kronecker products or sums of mode-wise attention matrices, enabling complexity control via factorization rank.

Result: HOT achieves competitive performance in multivariate time series forecasting and image classification on 2D/3D datasets with significantly reduced computational and memory costs, while learning interpretable high-order dependencies.

Conclusion: HOT provides an efficient and versatile framework for complex multiway data that preserves tensor structure while capturing high-order dependencies across diverse domains.

Abstract: Modern datasets are increasingly high-dimensional and multiway, often represented as tensor-valued data with multi-indexed variables. While Transformers excel in sequence modeling and high-dimensional tasks, their direct application to multiway data is computationally prohibitive due to the quadratic cost of dot-product attention and the need to flatten inputs, which disrupts tensor structure and cross-dimensional dependencies. We propose the Higher-Order Transformer (HOT), a novel factorized attention framework that represents multiway attention as sums of Kronecker products or sums of mode-wise attention matrices. HOT efficiently captures dense and sparse relationships across dimensions while preserving tensor structure. Theoretically, HOT retains the expressiveness of full high-order attention and allows complexity control via factorization rank. Experiments on 2D and 3D datasets show that HOT achieves competitive performance in multivariate time series forecasting and image classification, with significantly reduced computational and memory costs. Visualizations of mode-wise attention matrices further reveal interpretable high-order dependencies learned by HOT, demonstrating its versatility for complex multiway data across diverse domains. The implementation of our proposed method is publicly available at https://github.com/s-omranpour/HOT.

Method: Proposes DuCa (dual feature caching) that iteratively applies aggressive caching strategy (high caching ratio) and conservative caching strategy (low caching ratio), using random token selection instead of importance-based selection.

Result: Extensive experiments show DuCa significantly outperforms previous token-wise feature caching methods across DiT, PixArt, FLUX, and OpenSora models, demonstrating improved efficiency and performance.

Conclusion: The counter-intuitive findings reveal that consistent computation of ‘important’ tokens is unnecessary and importance-based selection can be ineffective, while DuCa’s dual caching with random selection provides superior acceleration for DiT-based models.

Abstract: Diffusion Transformers (DiT) have become the dominant methods in image and video generation yet still suffer substantial computational costs. As an effective approach for DiT acceleration, feature caching methods are designed to cache the features of DiT in previous timesteps and reuse them in the next timesteps, allowing us to skip the computation in the next timesteps. Among them, token-wise feature caching has been introduced to perform different caching ratios for different tokens in DiTs, aiming to skip the computation for unimportant tokens while still computing the important ones. In this paper, we propose to carefully check the effectiveness in token-wise feature caching with the following two questions: (1) Is it really necessary to compute the so-called “important” tokens in each step? (2) Are so-called important tokens really important? Surprisingly, this paper gives some counter-intuition answers, demonstrating that consistently computing the selected important tokens'' in all steps is not necessary. The selection of the so-called important tokens’’ is often ineffective, and even sometimes shows inferior performance than random selection. Based on these observations, this paper introduces dual feature caching referred to as DuCa, which performs aggressive caching strategy and conservative caching strategy iteratively and selects the tokens for computing randomly. Extensive experimental results demonstrate the effectiveness of our method in DiT, PixArt, FLUX, and OpenSora, demonstrating significant improvements than the previous token-wise feature caching.

Method: Examined AOL and SLL architectures, calculated exact and upper bounds for weight variance under Normal and Generalized Normal distributions, analyzed initialization effects on output variance.

Result: Weight variance has no bearing on output variance distribution; only weight matrix dimensions matter. Deep 1-Lipschitz networks always decay to zero due to weight initialization.

Conclusion: Current weight initialization methods for 1-Lipschitz networks are problematic as they cause network decay regardless of variance, highlighting the need for improved initialization strategies.

Abstract: This paper discusses the weight parametrization of two standard 1-Lipschitz network architectures, the Almost-Orthogonal-Layers (AOL) and the SDP-based Lipschitz Layers (SLL). It examines their impact on initialization for deep 1-Lipschitz feedforward networks, and discusses underlying issues surrounding this initialization. These networks are mainly used in certifiably robust classification applications to combat adversarial attacks by limiting the impact of perturbations on the classification output. Exact and upper bounds for the parameterized weight variance were calculated assuming a standard Normal distribution initialization; additionally, an upper bound was computed assuming a Generalized Normal Distribution, generalizing the proof for Uniform, Laplace, and Normal distribution weight initializations. It is demonstrated that the weight variance holds no bearing on the output variance distribution and that only the dimension of the weight matrices matters. Additionally, this paper demonstrates that the weight initialization always causes deep 1-Lipschitz networks to decay to zero.

Method: Introduces ElementaryNet, a novel neural network with mathematical guarantees that prevent it from expressing strategic behavior, while maintaining the ability to model non-strategic level-0 behavior.

Result: ElementaryNet and GameNet have statistically indistinguishable performance, demonstrating that the additional restrictions in ElementaryNet are empirically harmless while providing better interpretability.

Conclusion: ElementaryNet enables deriving insights about human behavior through feature variation and parameter interpretation, revealing evidence of iterative reasoning and the value of rich level-0 specifications.

Abstract: Behavioral game theory models serve two purposes: yielding insights into how human decision-making works, and predicting how people would behave in novel strategic settings. A system called GameNet represents the state of the art for predicting human behavior in the setting of unrepeated simultaneous-move games, combining a simple “level-k” model of strategic reasoning with a complex neural network model of non-strategic “level-0” behavior. Although this reliance on well-established ideas from cognitive science ought to make GameNet interpretable, the flexibility of its level-0 model raises the possibility that it is able to emulate strategic reasoning. In this work, we prove that GameNet’s level-0 model is indeed too general. We then introduce ElementaryNet, a novel neural network that is provably incapable of expressing strategic behavior. We show that these additional restrictions are empirically harmless, with ElementaryNet and GameNet having statistically indistinguishable performance. We then show how it is possible to derive insights about human behavior by varying ElementaryNet’s features and interpreting its parameters, finding evidence of iterative reasoning, learning about the depth of this reasoning process, and showing the value of a rich level-0 specification.

Method: Proposes ’equivarification’ - a general framework for modifying arbitrary neural networks to enforce equivariance properties.

Result: Demonstrates that group convolutional neural networks (G-CNNs) emerge as a specific instance within this broader equivarification framework.

Conclusion: Provides a unified theoretical foundation for equivariant neural networks, showing that existing approaches like G-CNNs are special cases of the proposed equivarification method.

Abstract: Equivariant neural networks are a class of neural networks designed to preserve symmetries inherent in the data. In this paper, we introduce a general method for modifying a neural network to enforce equivariance, a process we refer to as equivarification. We further show that group convolutional neural networks (G-CNNs) arise as a special case of our framework.

Method: Proposes a taxonomy categorizing CDGL approaches into structure-oriented, feature-oriented, and mixture-oriented based on the type of transferable knowledge learned across domains.

Result: Systematically summarizes representative methods in each category and provides a continuously updated collection of related works.

Conclusion: Identifies key challenges and limitations of current CDGL studies and outlines promising directions for future research toward developing graph foundation models.

Abstract: Graph learning plays a vital role in mining and analyzing complex relationships within graph data and has been widely applied to real-world scenarios such as social, citation, and e-commerce networks. Foundation models in computer vision (CV) and natural language processing (NLP) have demonstrated remarkable cross-domain capabilities that are equally significant for graph data. However, existing graph learning approaches often struggle to generalize across domains. Motivated by recent advances in CV and NLP, cross-domain graph learning (CDGL) has gained renewed attention as a promising step toward realizing true graph foundation models. In this survey, we provide a comprehensive review and analysis of existing works on CDGL. We propose a new taxonomy that categorizes existing approaches according to the type of transferable knowledge learned across domains: structure-oriented, feature-oriented, and mixture-oriented. Based on this taxonomy, we systematically summarize representative methods in each category, discuss the key challenges and limitations of current studies, and outline promising directions for future research. A continuously updated collection of related works is available at: https://github.com/cshhzhao/Awesome-Cross-Domain-Graph-Learning.

Method: Uses a direct high-dimensional semi-verified mean estimation method that identifies high-variance subspaces, estimates perpendicular components using corrupted gradients, and estimates subspace components using auxiliary clean datasets.

Result: Achieves minimax optimal statistical rates without the √d dependence on dimensionality, significantly outperforming existing methods by combining large corrupted datasets with small clean datasets.

Conclusion: The proposed method effectively addresses high-dimensional robust distributed learning problems, providing theoretical guarantees and practical effectiveness validated through numerical experiments.

Abstract: Adversarial attacks pose a major challenge to distributed learning systems, prompting the development of numerous robust learning methods. However, most existing approaches suffer from the curse of dimensionality, i.e. the error increases with the number of model parameters. In this paper, we make a progress towards high dimensional problems, under arbitrary number of Byzantine attackers. The cornerstone of our design is a direct high dimensional semi-verified mean estimation method. The idea is to identify a subspace with large variance. The components of the mean value perpendicular to this subspace are estimated using corrupted gradient vectors uploaded from worker machines, while the components within this subspace are estimated using auxiliary dataset. As a result, a combination of large corrupted dataset and small clean dataset yields significantly better performance than using them separately. We then apply this method as the aggregator for distributed learning problems. The theoretical analysis shows that compared with existing solutions, our method gets rid of $\sqrt{d}$ dependence on the dimensionality, and achieves minimax optimal statistical rates. Numerical results validate our theory as well as the effectiveness of the proposed method.

Method: Proposes Non-Uniform Class-Wise Coreset Selection (NUCS) with a robust global class difficulty metric using winsorized average of per-sample scores, enabling theoretically-grounded non-uniform inter-class budget allocation and adaptive intra-class sample selection.

Result: Extensive experiments on 10 diverse datasets and pre-trained models show NUCS consistently outperforms state-of-the-art methods in both accuracy and computational efficiency.

Conclusion: Non-uniform class-wise selection strategy shows promise for advancing efficient fine-tuning of large foundation models.

Abstract: Coreset selection aims to identify a small yet highly informative subset of data, thereby enabling more efficient model training while reducing storage overhead. Recently, this capability has been leveraged to tackle the challenges of fine-tuning large foundation models, offering a direct pathway to their efficient and practical deployment. However, most existing methods are class-agnostic, causing them to overlook significant difficulty variations among classes. This leads them to disproportionately prune samples from either overly easy or hard classes, resulting in a suboptimal allocation of the data budget that ultimately degrades the final coreset performance. To address this limitation, we propose Non-Uniform Class-Wise Coreset Selection (NUCS), a novel framework that both integrates class-level and sample-level difficulty. We propose a robust metric for global class difficulty, quantified as the winsorized average of per-sample difficulty scores. Guided by this metric, our method performs a theoretically-grounded, non-uniform allocation of data selection budgets inter-class, while adaptively selecting samples intra-class with optimal difficulty ranges. Extensive experiments on a wide range of visual classification tasks demonstrate that NUCS consistently outperforms state-of-the-art methods across 10 diverse datasets and pre-trained models, achieving both superior accuracy and computational efficiency, highlighting the promise of non-uniform class-wise selection strategy for advancing the efficient fine-tuning of large foundation models.

Method: The authors provide theoretical analysis of sample complexity for learning accurate diffusion models using expressive neural networks, focusing on improved parameter dependencies.

Result: The paper demonstrates exponential improvement in dependence on Wasserstein error and depth parameters, along with better dependencies on other relevant parameters compared to previous bounds.

Conclusion: The work establishes significantly tighter sample complexity bounds for training diffusion models, providing stronger theoretical foundations for their empirical success.

Abstract: Diffusion models have become the most popular approach to deep generative modeling of images, largely due to their empirical performance and reliability. From a theoretical standpoint, a number of recent works have studied the iteration complexity of sampling, assuming access to an accurate diffusion model. In this work, we focus on understanding the sample complexity of training such a model; how many samples are needed to learn an accurate diffusion model using a sufficiently expressive neural network? Prior work showed bounds polynomial in the dimension, desired Total Variation error, and Wasserstein error. We show an exponential improvement in the dependence on Wasserstein error and depth, along with improved dependencies on other relevant parameters.

Method: Develops a new framework operating in primal space that resolves the primal LP for CMDP problems online with adaptive remaining resource capacities. Key elements include: instance hardness characterization via LP basis, an eliminating procedure to identify optimal basis, and a resolving procedure adaptive to remaining resources.

Result: Achieves logarithmic regret bound and improved sample complexity O(1/(Δ·ε)·log²(1/ε)), where Δ is a problem-dependent parameter independent of ε, significantly improving upon the state-of-art O(1/ε²) sample complexity.

Conclusion: The proposed framework provides the first optimal problem-dependent guarantees for CMDP problems, with substantial improvements in sample complexity dependency on ε through novel primal space analysis and adaptive LP resolution techniques.

Abstract: We consider the reinforcement learning problem for the constrained Markov decision process (CMDP), which plays a central role in satisfying safety or resource constraints in sequential learning and decision-making. In this problem, we are given finite resources and a MDP with unknown transition probabilities. At each stage, we take an action, collecting a reward and consuming some resources, all assumed to be unknown and need to be learned over time. In this work, we take the first step towards deriving optimal problem-dependent guarantees for the CMDP problems. We derive a logarithmic regret bound, which translates into a $O(\frac{1}{Δ\cdotε}\cdot\log^2(1/ε))$ sample complexity bound, with $Δ$ being a problem-dependent parameter, yet independent of $ε$. Our sample complexity bound improves upon the state-of-art $O(1/ε^2)$ sample complexity for CMDP problems established in the previous literature, in terms of the dependency on $ε$. To achieve this advance, we develop a new framework for analyzing CMDP problems. To be specific, our algorithm operates in the primal space and we resolve the primal LP for the CMDP problem at each period in an online manner, with adaptive remaining resource capacities. The key elements of our algorithm are: i) a characterization of the instance hardness via LP basis, ii) an eliminating procedure that identifies one optimal basis of the primal LP, and; iii) a resolving procedure that is adaptive to the remaining resources and sticks to the characterized optimal basis.

Method: Used DolusChat dataset with truthful/deceptive response pairs, tested GRPO and DPO algorithms with lie detectors, varying exploration, detector accuracy (TPR), and KL regularization strength.

Result: GRPO with lie detectors can produce policies that evade detection with 85%+ deception rates, but with high TPR or strong KL regularization, learns honest policies. DPO consistently achieves <25% deception for realistic TPRs.

Conclusion: Lie-detector-enhanced training has complex outcomes - can enable scalable oversight or encourage undetectable misalignment depending on context factors like algorithm choice and detector quality.

Abstract: As AI systems become more capable, deceptive behaviors can undermine evaluation and mislead users at deployment. Recent work has shown that lie detectors can accurately classify deceptive behavior, but they are not typically used in the training pipeline due to concerns around contamination and objective hacking. We examine these concerns by incorporating a lie detector into the labelling step of LLM post-training and evaluating whether the learned policy is genuinely more honest, or instead learns to fool the lie detector while remaining deceptive. Using DolusChat, a novel 65k-example dataset with paired truthful/deceptive responses, we identify three key factors that determine the honesty of learned policies: amount of exploration during preference learning, lie detector accuracy, and KL regularization strength. We find that preference learning with lie detectors and GRPO can lead to policies which evade lie detectors, with deception rates of over 85%. However, if the lie detector true positive rate (TPR) or KL regularization is sufficiently high, GRPO learns honest policies. In contrast, off-policy algorithms (DPO) consistently lead to deception rates under 25% for realistic TPRs. Our results illustrate a more complex picture than previously assumed: depending on the context, lie-detector-enhanced training can be a powerful tool for scalable oversight, or a counterproductive method encouraging undetectable misalignment.

Method: Proposes PRIMUS with a novel pretraining objective that combines self-supervision, multimodal supervision, and nearest-neighbor supervision to build strong IMU feature extractors.

Result: With fewer than 500 labeled samples per class, PRIMUS improves test accuracy by up to 15% compared to state-of-the-art baselines on both in-domain and out-of-domain datasets.

Conclusion: PRIMUS effectively addresses the scarcity of labeled IMU data through pretraining and enables better performance on human motion tasks with limited labeled samples.

Abstract: Sensing human motions through Inertial Measurement Units (IMUs) embedded in personal devices has enabled significant applications in health and wellness. Labeled IMU data is scarce, however, unlabeled or weakly labeled IMU data can be used to model human motions. For video or text modalities, the “pretrain and adapt” approach utilizes large volumes of unlabeled or weakly labeled data to build a strong feature extractor, followed by adaptation to specific tasks using limited labeled data. However, pretraining methods are poorly understood for IMU data, and pipelines are rarely evaluated on out-of-domain tasks. We propose PRIMUS: a method for PRetraining IMU encoderS that uses a novel pretraining objective that is empirically validated based on downstream performance on both in-domain and out-of-domain datasets. The PRIMUS objective effectively enhances downstream performance by combining self-supervision, multimodal, and nearest-neighbor supervision. With fewer than 500 labeled samples per class, PRIMUS improves test accuracy by up to 15%, compared to state-of-the-art baselines. To benefit the broader community, we have open-sourced our code at github.com/nokia-bell-labs/pretrained-imu-encoders.

Method: Pretrain transformer-based neural network on synthetic data, then fine-tune on small experimental dataset (64 spectra). Tested in simulation-to-simulation and simulation-to-experimental scenarios.

Result: Achieved 96% test accuracy with LaBr detector in sim-to-real scenario, outperforming synthetic-only baseline (75%) and from-scratch training (80%). Models also learned more interpretable features.

Conclusion: Supervised domain adaptation enables accurate and explainable radioisotope classifiers even with limited experimental data, effectively bridging the sim-to-real gap.

Abstract: Machine learning has the potential to improve the speed and reliability of radioisotope identification using gamma spectroscopy. However, meticulously labeling an experimental dataset for training is often prohibitively expensive, while training models purely on synthetic data is risky due to the domain gap between simulated and experimental measurements. In this research, we demonstrate that supervised domain adaptation can substantially improve the performance of radioisotope identification models by transferring knowledge between synthetic and experimental data domains. We consider two domain adaptation scenarios: (1) a simulation-to-simulation adaptation, where we perform multi-label proportion estimation using simulated high-purity germanium detectors, and (2) a simulation-to-experimental adaptation, where we perform multi-class, single-label classification using measured spectra from handheld lanthanum bromide (LaBr) and sodium iodide (NaI) detectors. We begin by pretraining a spectral classifier on synthetic data using a custom transformer-based neural network. After subsequent fine-tuning on just 64 labeled experimental spectra, we achieve a test accuracy of 96% in the sim-to-real scenario with a LaBr detector, far surpassing a synthetic-only baseline model (75%) and a model trained from scratch (80%) on the same 64 spectra. Furthermore, we demonstrate that domain-adapted models learn more human-interpretable features than experiment-only baseline models. Overall, our results highlight the potential for supervised domain adaptation techniques to bridge the sim-to-real gap in radioisotope identification, enabling the development of accurate and explainable classifiers even in real-world scenarios where access to experimental data is limited.

Method: FairDICE uses distribution correction estimation to jointly account for welfare maximization and distributional regularization, enabling stable learning without explicit preference weights or exhaustive weight search.

Result: Across multiple offline benchmarks, FairDICE demonstrates strong fairness-aware performance compared to existing baselines.

Conclusion: FairDICE provides a unified approach for optimizing nonlinear welfare criteria in offline MORL, enabling sample-efficient learning of fairness-oriented policies from fixed datasets.

Abstract: Multi-objective reinforcement learning (MORL) aims to optimize policies in the presence of conflicting objectives, where linear scalarization is commonly used to reduce vector-valued returns into scalar signals. While effective for certain preferences, this approach cannot capture fairness-oriented goals such as Nash social welfare or max-min fairness, which require nonlinear and non-additive trade-offs. Although several online algorithms have been proposed for specific fairness objectives, a unified approach for optimizing nonlinear welfare criteria in the offline setting-where learning must proceed from a fixed dataset-remains unexplored. In this work, we present FairDICE, the first offline MORL framework that directly optimizes nonlinear welfare objective. FairDICE leverages distribution correction estimation to jointly account for welfare maximization and distributional regularization, enabling stable and sample-efficient learning without requiring explicit preference weights or exhaustive weight search. Across multiple offline benchmarks, FairDICE demonstrates strong fairness-aware performance compared to existing baselines.

Method: Machine learning-based modeling with an extended set of features, using rigorous statistical assessment and test set holdouts to prove model generalization.

Result: The extended feature set improves prediction accuracy while demonstrating robust model generalization capabilities.

Conclusion: Feature-based machine learning approaches enable accurate, efficient, and scalable propagation modeling for wireless communications when combined with high-resolution geographic data and proper statistical validation.

Abstract: Wireless communications rely on path loss modeling, which is most effective when it includes the physical details of the propagation environment. Acquiring this data has historically been challenging, but geographic information systems data is becoming increasingly available with higher resolution and accuracy. Access to such details enables propagation models to more accurately predict coverage and account for interference in wireless deployments. Machine learning-based modeling can significantly support this effort, with feature based approaches allowing for accurate, efficient, and scalable propagation modeling. Building on previous work, we introduce an extended set of features that improves prediction accuracy while, most importantly, proving model generalization through rigorous statistical assessment and the use of test set holdouts.

Method: Uses only minibatches sampled under importance sampling to estimate variance reduction, introduces an effective minibatch size for automatic learning rate adjustment, and develops real-time importance score estimation based on moving gradient statistics.

Result: Theoretical analysis and experiments show consistent variance reduction, improved training efficiency, and enhanced model accuracy compared to current importance-sampling approaches with minimal computational overhead.

Conclusion: The proposed algorithm effectively improves DNN training efficiency through better variance reduction estimation and automatic learning rate adjustment while maintaining low computational cost.

Abstract: Importance sampling is widely used to improve the efficiency of deep neural network (DNN) training by reducing the variance of gradient estimators. However, efficiently assessing the variance reduction relative to uniform sampling remains challenging due to computational overhead. This paper proposes a method for estimating variance reduction during DNN training using only minibatches sampled under importance sampling. By leveraging the proposed method, the paper also proposes an effective minibatch size to enable automatic learning rate adjustment. An absolute metric to quantify the efficiency of importance sampling is also introduced as well as an algorithm for real-time estimation of importance scores based on moving gradient statistics. Theoretical analysis and experiments on benchmark datasets demonstrated that the proposed algorithm consistently reduces variance, improves training efficiency, and enhances model accuracy compared with current importance-sampling approaches while maintaining minimal computational overhead.

Method: FP generates target values for pre-activation membrane potentials through randomized nonlinear projections of pre-synaptic inputs and labels, then optimizes local loss functions using closed-form regression without feedback from downstream layers.

Result: FP achieves comparable generalization to gradient descent-based local learning methods on large-sample tasks, and in some few-shot learning tasks, yields more generalizable models than backpropagation. Training requires only a single forward pass, providing significant speedup.

Conclusion: FP is an effective backpropagation-free training method that enables interpretable layer-wise predictions and achieves competitive performance with substantial training efficiency improvements, particularly suitable for scenarios where retrograde communication is unavailable.

Abstract: State-of-the-art methods for backpropagation-free learning employ local error feedback to direct iterative optimisation via gradient descent. In this study, we examine the more restrictive setting where retrograde communication from neuronal outputs is unavailable for pre-synaptic weight optimisation. To address this challenge, we propose Forward Projection (FP). This randomised closed-form training method requires only a single forward pass over the entire dataset for model fitting, without retrograde communication. Our method generates target values for pre-activation membrane potentials at each layer through randomised nonlinear projections of pre-synaptic inputs and the labels, thereby encoding information from both sources. Local loss functions are optimised over pre-synaptic inputs using closed-form regression, without feedback from neuronal outputs or downstream layers. Interpretability is a key advantage of FP training; membrane potentials of hidden neurons in FP-trained networks encode information which are interpretable layer-wise as label predictions. We demonstrate the effectiveness of FP across four biomedical datasets, comparing it with backpropagation and local learning techniques such as Forward-Forward training and Local Supervision in multi-layer perceptron and convolutional architectures. In some few-shot learning tasks, FP yielded more generalisable models than those optimised via backpropagation. In large-sample tasks, FP-based models achieve generalisation comparable to gradient descent-based local learning methods while requiring only a single forward propagation step, achieving significant speed up for training.

Method: SPLIT algorithms use sparse lookahead with dynamic programming, solving only key sub-problems optimally while employing greediness near tree leaves to reduce computational complexity.

Result: Orders of magnitude faster than optimal methods with negligible performance loss, enabling scalable computation of near-optimal tree sets (Rashomon set).

Conclusion: SPLIT achieves near-optimal accuracy with greedy-level scalability, significantly advancing interpretable machine learning.

Abstract: Decision tree optimization is fundamental to interpretable machine learning. The most popular approach is to greedily search for the best feature at every decision point, which is fast but provably suboptimal. Recent approaches find the global optimum using branch and bound with dynamic programming, showing substantial improvements in accuracy and sparsity at great cost to scalability. An ideal solution would have the accuracy of an optimal method and the scalability of a greedy method. We introduce a family of algorithms called SPLIT (SParse Lookahead for Interpretable Trees) that moves us significantly forward in achieving this ideal balance. We demonstrate that not all sub-problems need to be solved to optimality to find high quality trees; greediness suffices near the leaves. Since each depth adds an exponential number of possible trees, this change makes our algorithms orders of magnitude faster than existing optimal methods, with negligible loss in performance. We extend this algorithm to allow scalable computation of sets of near-optimal trees (i.e., the Rashomon set).

Method: Defines robust hollow star number to characterize certificate sizes, analyzes worst-case and distributional bounds, and introduces certificate coefficient for sample size analysis.

Result: Shows robust hollow star number precisely characterizes smallest certificate size, provides matching bounds on sample size as function of certificate coefficient, corruption bound, and VC dimension.

Conclusion: The framework provides theoretically grounded explanations with provable guarantees, with certificate sizes characterized by robust hollow star number and sample complexity controlled by certificate coefficient.

Abstract: We consider a model for explainable AI in which an explanation for a prediction $h(x)=y$ consists of a subset $S’$ of the training data (if it exists) such that all classifiers $h’ \in H$ that make at most $b$ mistakes on $S’$ predict $h’(x)=y$. Such a set $S’$ serves as a proof that $x$ indeed has label $y$ under the assumption that (1) the target function $h^\star$ belongs to $H$, and (2) the set $S$ contains at most $b$ corrupted points. For example, if $b=0$ and $H$ is the family of linear classifiers in $\mathbb{R}^d$, and if $x$ lies inside the convex hull of the positive data points in $S$ (and hence every consistent linear classifier labels $x$ as positive), then Carathéodory’s theorem states that $x$ lies inside the convex hull of $d+1$ of those points. So, a set $S’$ of size $d+1$ could be released as an explanation for a positive prediction, and would serve as a short proof of correctness of the prediction under the assumption of realizability. In this work, we consider this problem more generally, for general hypothesis classes $H$ and general values $b\geq 0$. We define the notion of the robust hollow star number of $H$ (which generalizes the standard hollow star number), and show that it precisely characterizes the worst-case size of the smallest certificate achievable, and analyze its size for natural classes. We also consider worst-case distributional bounds on certificate size, as well as distribution-dependent bounds that we show tightly control the sample size needed to get a certificate for any given test example. In particular, we define a notion of the certificate coefficient $\varepsilon_x$ of an example $x$ with respect to a data distribution $D$ and target function $h^\star$, and prove matching upper and lower bounds on sample size as a function of $\varepsilon_x$, $b$, and the VC dimension $d$ of $H$.

Method: Uses a 565M-parameter latent diffusion model trained on ERA5 data to perform score-based data assimilation, generating global atmospheric trajectories at 0.25° resolution and 1-hour intervals without requiring retraining for different observation types.

Result: The model successfully handles reanalysis, filtering, and forecasting within a single framework, producing physically consistent atmospheric reconstructions from various input observations.

Conclusion: Latent score-based data assimilation represents a promising foundation for future global atmospheric modeling systems, offering a unified probabilistic approach to atmospheric state estimation.

Abstract: Deep learning has advanced weather forecasting, but accurate predictions first require identifying the current state of the atmosphere from observational data. In this work, we introduce Appa, a score-based data assimilation model generating global atmospheric trajectories at 0.25\si{\degree} resolution and 1-hour intervals. Powered by a 565M-parameter latent diffusion model trained on ERA5, Appa can be conditioned on arbitrary observations to infer plausible trajectories, without retraining. Our probabilistic framework handles reanalysis, filtering, and forecasting, within a single model, producing physically consistent reconstructions from various inputs. Results establish latent score-based data assimilation as a promising foundation for future global atmospheric modeling systems.

Method: Used tools from dynamical systems theory to analyze linear networks, examining how eigenvalues of input correlation matrix, initial weight scale relative to ground-truth output, and number of gradient descent steps control manifold geometry.

Result: Characterized phase boundaries where hyper-ribbons are expected, showing the geometry is controlled by: (i) eigenvalue decay of input correlation matrix, (ii) relative scale of ground-truth output to initial weights, (iii) number of gradient steps. Extended analysis to kernel machines and linear models with SGD.

Conclusion: Successfully provided analytical characterization of the low-dimensional manifold phenomenon in neural network training trajectories, identifying key controlling factors and extending the analysis to broader machine learning models.

Abstract: Recent experiments have shown that training trajectories of multiple deep neural networks with different architectures, optimization algorithms, hyper-parameter settings, and regularization methods evolve on a remarkably low-dimensional “hyper-ribbon-like” manifold in the space of probability distributions. Inspired by the similarities in the training trajectories of deep networks and linear networks, we analytically characterize this phenomenon for the latter. We show, using tools in dynamical systems theory, that the geometry of this low-dimensional manifold is controlled by (i) the decay rate of the eigenvalues of the input correlation matrix of the training data, (ii) the relative scale of the ground-truth output to the weights at the beginning of training, and (iii) the number of steps of gradient descent. By analytically computing and bounding the contributions of these quantities, we characterize phase boundaries of the region where hyper-ribbons are to be expected. We also extend our analysis to kernel machines and linear models that are trained with stochastic gradient descent.

Method: Introduces Quartet - an optimized FP4 training approach using hardware-supported FP4 operations with tailored CUDA kernels for Blackwell architecture, guided by a new low-precision scaling law.

Result: Demonstrates that fully FP4-based training is competitive with FP16 half-precision and FP8 training, achieving accurate end-to-end training with all major computations in low precision.

Conclusion: FP4 training with Quartet is a viable and efficient alternative for LLM training, offering significant computational benefits while maintaining model accuracy comparable to higher-precision methods.

Abstract: Training large language models (LLMs) models directly in low-precision offers a way to address computational costs by improving both throughput and energy efficiency. For those purposes, NVIDIA’s recent Blackwell architecture facilitates very low-precision operations using FP4 variants. Yet, current algorithms for training LLMs in FP4 precision face significant accuracy degradation and often rely on mixed-precision fallbacks. In this paper, we investigate hardware-supported FP4 training and introduce a new approach for accurate, end-to-end FP4 training with all the major computations (i.e., linear layers) in low precision. Through extensive evaluations on Llama-type models, we reveal a new low-precision scaling law that quantifies performance trade-offs across bit-widths and training setups. Guided by this investigation, we design an “optimal” technique in terms of accuracy-vs-computation, called Quartet. We implement Quartet using optimized CUDA kernels tailored for Blackwell, demonstrating that fully FP4-based training is a competitive alternative to FP16 half-precision and to FP8 training. Our code is available at https://github.com/IST-DASLab/Quartet.

Method: Uses multimodal contrastive learning to align IMU data with natural language descriptions, enabling few-shot sensitive activity detection. Includes language-guided sanitizer and IMU-GPT motion generation to transform sensitive data into privacy-compliant versions.

Result: Significantly outperforms baseline methods on multiple human activity recognition datasets in both privacy protection and data utility.

Conclusion: PrivCLIP provides effective dynamic privacy control for IMU-based sensing systems with minimal training data requirements.

Abstract: User-controllable privacy is important in modern sensing systems, as privacy preferences can vary significantly from person to person and may evolve over time. This is especially relevant in devices equipped with Inertial Measurement Unit (IMU) sensors, such as smartphones and wearables, which continuously collect rich time-series data that can inadvertently expose sensitive user behaviors. While prior work has proposed privacy-preserving methods for sensor data, most rely on static, predefined privacy labels or require large quantities of private training data, limiting their adaptability and user agency. In this work, we introduce PrivCLIP, a dynamic, user-controllable, few-shot privacy-preserving sensing framework. PrivCLIP allows users to specify and modify their privacy preferences by categorizing activities as sensitive (black-listed), non-sensitive (white-listed), or neutral (gray-listed). Leveraging a multimodal contrastive learning approach, PrivCLIP aligns IMU sensor data with natural language activity descriptions in a shared embedding space, enabling few-shot detection of sensitive activities. When a privacy-sensitive activity is identified, the system uses a language-guided activity sanitizer and a motion generation module (IMU-GPT) to transform the original data into a privacy-compliant version that semantically resembles a non-sensitive activity. We evaluate PrivCLIP on multiple human activity recognition datasets and demonstrate that it significantly outperforms baseline methods in terms of both privacy protection and data utility.

Method: Uses tools from statistical mechanics and random matrix theory to derive closed-form predictions for Bayes-optimal generalization error, and proposes a matching approximate message passing algorithm.

Result: Identifies sharp phase transitions as a function of sample complexity, model width, and sequence length, and shows that gradient descent can reach optimal performance.

Conclusion: AIM offers a solvable playground for understanding learning in self-attention layers, providing theoretical insights into deep attention mechanisms.

Abstract: We introduce the attention-indexed model (AIM), a theoretical framework for analyzing learning in deep attention layers. Inspired by multi-index models, AIM captures how token-level outputs emerge from layered bilinear interactions over high-dimensional embeddings. Unlike prior tractable attention models, AIM allows full-width key and query matrices, aligning more closely with practical transformers. Using tools from statistical mechanics and random matrix theory, we derive closed-form predictions for Bayes-optimal generalization error and identify sharp phase transitions as a function of sample complexity, model width, and sequence length. We propose a matching approximate message passing algorithm and show that gradient descent can reach optimal performance. AIM offers a solvable playground for understanding learning in self-attention layers, that are key components of modern architectures.

Method: Trains a universal multi-scale autoencoder to embed architectures and weights into a continuous latent space, then optimizes points in this space via gradient descent with sparsity/compactness penalties.

Result: Successfully discovers sparse and compact neural networks with strong performance on synthetic regression tasks.

Conclusion: The proposed joint optimization approach effectively discovers efficient neural networks by simultaneously optimizing architecture and weights in a continuous embedding space.

Abstract: Designing neural networks typically relies on manual trial and error or a neural architecture search (NAS) followed by weight training. The former is time-consuming and labor-intensive, while the latter often discretizes architecture search and weight optimization. In this paper, we propose a fundamentally different approach that simultaneously optimizes both the architecture and the weights of a neural network. Our framework first trains a universal multi-scale autoencoder that embeds both architectural and parametric information into a continuous latent space, where functionally similar neural networks are mapped closer together. Given a dataset, we then randomly initialize a point in the embedding space and update it via gradient descent to obtain the optimal neural network, jointly optimizing its structure and weights. The optimization process incorporates sparsity and compactness penalties to promote efficient models. Experiments on synthetic regression tasks demonstrate that our method effectively discovers sparse and compact neural networks with strong performance.

Method: Rewire transformer blocks to solve an inner discretized ODE with respect to depth, apply length consistency term during flow matching training, enabling sampling with arbitrary time steps and transformer blocks in a solver-agnostic approach.

Result: Achieves up to 2× latency reduction in efficient sampling mode and up to 2.8 FID improvement for high-quality sampling on CelebA-HQ and ImageNet, while reducing both latency and memory usage.

Conclusion: The ODE_t(ODE_l) approach provides a complementary direction to time step reduction, offering solver-agnostic efficiency improvements that reduce both computational latency and memory requirements in generative models.

Abstract: Continuous normalizing flows (CNFs) and diffusion models (DMs) generate high-quality data from a noise distribution. However, their sampling process demands multiple iterations to solve an ordinary differential equation (ODE) with high computational complexity. State-of-the-art methods focus on reducing the number of discrete time steps during sampling to improve efficiency. In this work, we explore a complementary direction in which the quality-complexity tradeoff can also be controlled in terms of the neural network length. We achieve this by rewiring the blocks in the transformer-based architecture to solve an inner discretized ODE w.r.t. its depth. Then, we apply a length consistency term during flow matching training, and as a result, the sampling can be performed with an arbitrary number of time steps and transformer blocks. Unlike others, our ODE$_t$(ODE$_l$) approach is solver-agnostic in time dimension and reduces both latency and, importantly, memory usage. CelebA-HQ and ImageNet generation experiments show a latency reduction of up to $2\times$ in the most efficient sampling mode, and FID improvement of up to $2.8$ points for high-quality sampling when applied to prior methods. We open-source our code and checkpoints at github.com/gudovskiy/odelt.

Method: SEPAL ensures global embedding consistency by optimizing embeddings only on a small core of entities and then propagating them to the rest of the graph using message passing.

Result: SEPAL significantly outperforms previous methods on 46 downstream machine learning tasks across 7 large-scale knowledge graphs, and scales up its base embedding model to fit huge knowledge graphs on commodity hardware.

Conclusion: SEPAL addresses scalability and downstream task performance limitations of current knowledge graph embedding methods through its core optimization and propagation approach, demonstrating superior performance and practical deployment advantages.

Abstract: Many machine learning tasks can benefit from external knowledge. Large knowledge graphs store such knowledge, and embedding methods can be used to distill it into ready-to-use vector representations for downstream applications. For this purpose, current models have however two limitations: they are primarily optimized for link prediction, via local contrastive learning, and their application to the largest graphs requires significant engineering effort due to GPU memory limits. To address these, we introduce SEPAL: a Scalable Embedding Propagation ALgorithm for large knowledge graphs designed to produce high-quality embeddings for downstream tasks at scale. The key idea of SEPAL is to ensure global embedding consistency by optimizing embeddings only on a small core of entities, and then propagating them to the rest of the graph with message passing. We evaluate SEPAL on 7 large-scale knowledge graphs and 46 downstream machine learning tasks. Our results show that SEPAL significantly outperforms previous methods on downstream tasks. In addition, SEPAL scales up its base embedding model, enabling fitting huge knowledge graphs on commodity hardware.

Method: Two configurations: (1) Heisenberg diffusion model that compensates principal axis scaling with inverse scaling on orthogonal directions, (2) standard SDM that scales only the principal axis. Tested on CIFAR-10/100 and CelebA-64 datasets.

Result: Counterintuitively, mild antisqueezing (increasing variance on principal axis) consistently improves FID by up to 15% and shifts precision-recall frontier toward higher recall.

Conclusion: Simple, data-aware noise shaping can deliver robust generative gains without requiring architectural modifications to diffusion models.

Abstract: Diffusion models typically inject isotropic Gaussian noise, disregarding structure in the data. Motivated by the way quantum squeezed states redistribute uncertainty according to the Heisenberg uncertainty principle, we introduce Squeezed Diffusion Models (SDM), which scale noise anisotropically along the principal component of the training distribution. As squeezing enhances the signal-to-noise ratio in physics, we hypothesize that scaling noise in a data-dependent manner can better assist diffusion models in learning important data features. We study two configurations: (i) a Heisenberg diffusion model that compensates the scaling on the principal axis with inverse scaling on orthogonal directions and (ii) a standard SDM variant that scales only the principal axis. Counterintuitively, on CIFAR-10/100 and CelebA-64, mild antisqueezing - i.e. increasing variance on the principal axis - consistently improves FID by up to 15% and shifts the precision-recall frontier toward higher recall. Our results demonstrate that simple, data-aware noise shaping can deliver robust generative gains without architectural changes.

Method: Proposes DPO-PRO, a robust fine-tuning algorithm based on Direct Preference Optimization (DPO) that incorporates a lightweight Distributionally Robust Optimization (DRO) formulation to account for uncertainty in the preference distribution.

Result: DPO-PRO consistently improves robustness to noisy preference signals compared to existing DPO variants, achieves comparable performance to self-reflection-based baselines for reward function design, and requires significantly lower inference-time costs.

Conclusion: DPO-PRO provides an effective and efficient solution for robust LLM fine-tuning in complex sequential resource allocation problems, particularly in public health applications with limited data and ambiguous objectives.

Abstract: We study an LLM fine-tuning task for designing reward functions for sequential resource allocation problems in public health, guided by human preferences expressed in natural language. This setting presents a challenging testbed for alignment due to complex and ambiguous objectives and limited data availability. We propose DPO-PRO, a robust fine-tuning algorithm based on Direct Preference Optimization (DPO), which accounts for uncertainty in the preference distribution using a lightweight Distributionally Robust Optimization (DRO) formulation. Unlike prior DRO-based DPO methods, DPO-PRO is significantly less conservative. We evaluate DPO-PRO on a real-world maternal mobile health program operated by the non-profit organization ARMMAN, as well as on standard alignment benchmarks. Experimental results demonstrate that our method consistently improves robustness to noisy preference signals compared to existing DPO variants. Moreover, DPO-PRO achieves comparable performance to prior self-reflection-based baseline for reward function design, while requiring significantly lower inference-time cost.

Method: Employ the formal verification tool Marabou to verify local robustness of neural network models, quantifying how much input can be perturbed before output behavior becomes unstable.

Result: Improved trustworthiness of AI-based fault detection systems under uncertainty by formally verifying neural network robustness.

Conclusion: Formal verification using Marabou enables development of highly reliable embedded AI systems for satellite fault detection by ensuring neural network robustness against input perturbations.

Abstract: Failures in satellite components are costly and challenging to address, often requiring significant human and material resources. Embedding a hybrid AI-based system for fault detection directly in the satellite can greatly reduce this burden by allowing earlier detection. However, such systems must operate with extremely high reliability. To ensure this level of dependability, we employ the formal verification tool Marabou to verify the local robustness of the neural network models used in the AI-based algorithm. This tool allows us to quantify how much a model’s input can be perturbed before its output behavior becomes unstable, thereby improving trustworthiness with respect to its performance under uncertainty.

Method: The authors provide a theoretical characterization of unfairness in algorithmic recourse, formally link fairness guarantees between recourse and classification, and introduce a novel fairness framework based on social burden. They develop a practical algorithm called MISOB that is applicable under real-world conditions.

Result: Empirical evaluation on real-world datasets demonstrates that MISOB successfully reduces social burden across all demographic groups while maintaining overall classifier accuracy.

Conclusion: The proposed social burden framework and MISOB algorithm provide an effective approach to ensuring fairness in algorithmic recourse, addressing limitations of existing equal cost paradigms and offering practical solutions for real-world deployment.

Abstract: Machine learning based predictions are increasingly used in sensitive decision-making applications that directly affect our lives. This has led to extensive research into ensuring the fairness of classifiers. Beyond just fair classification, emerging legislation now mandates that when a classifier delivers a negative decision, it must also offer actionable steps an individual can take to reverse that outcome. This concept is known as algorithmic recourse. Nevertheless, many researchers have expressed concerns about the fairness guarantees within the recourse process itself. In this work, we provide a holistic theoretical characterization of unfairness in algorithmic recourse, formally linking fairness guarantees in recourse and classification, and highlighting limitations of the standard equal cost paradigm. We then introduce a novel fairness framework based on social burden, along with a practical algorithm (MISOB), broadly applicable under real-world conditions. Empirical results on real-world datasets show that MISOB reduces the social burden across all groups without compromising overall classifier accuracy.

Method: Introduces cutmax - an HE-friendly argmax algorithm that reduces ciphertext operations, and the first HE-compatible nucleus (top-p) sampling method. Both techniques are polynomial and differentiable, supporting efficient inference and gradient-based optimization.

Result: Achieves latency reductions of 24x-35x over baseline methods on realistic LLM outputs while maintaining provable privacy guarantees and enabling practical secure text generation.

Conclusion: Cutmax advances secure text generation by making greedy and stochastic decoding practical under homomorphic encryption, with strong theoretical convergence guarantees and significant performance improvements.

Abstract: Large language models (LLMs) power modern AI applications, but processing sensitive data on untrusted servers raises privacy concerns. Homomorphic encryption (HE) enables computation on encrypted data for secure inference. However, neural text generation requires decoding methods like argmax and sampling, which are non-polynomial and thus computationally expensive under encryption, creating a significant performance bottleneck. We introduce cutmax, an HE-friendly argmax algorithm that reduces ciphertext operations compared to prior methods, enabling practical greedy decoding under encryption. We also propose the first HE-compatible nucleus (top-p) sampling method, leveraging cutmax for efficient stochastic decoding with provable privacy guarantees. Both techniques are polynomial, supporting efficient inference in privacy-preserving settings. Moreover, their differentiability facilitates gradient-based sequence-level optimization as a polynomial alternative to straight-through estimators. We further provide strong theoretical guarantees for cutmax, proving its convergence via exponential amplification of the gap ratio between the maximum and runner-up elements. Evaluations on realistic LLM outputs show latency reductions of 24x-35x over baselines, advancing secure text generation.

Method: Introduces a contextual learning framework with: (1) probabilistic formulation for context-conditioned learning, (2) bilevel optimization for automatic context feature selection using validation loss, and (3) conditional Wasserstein autoencoder for modeling conditional distributions P(Y|C).

Result: Extensive experiments across eight benchmark datasets demonstrate that contextual learning consistently outperforms global approaches, even when optimal context isn’t intuitively obvious.

Conclusion: The framework establishes a new foundation for anomaly detection in heterogeneous tabular data by explicitly modeling how normal behavior varies across different contexts.

Abstract: Anomaly detection is critical in domains such as cybersecurity and finance, especially when working with large-scale tabular data. Yet, unsupervised anomaly detection-where no labeled anomalies are available-remains challenging because traditional deep learning methods model a single global distribution, assuming all samples follow the same behavior. In contrast, real-world data often contain heterogeneous contexts (e.g., different users, accounts, or devices), where globally rare events may be normal within specific conditions. We introduce a contextual learning framework that explicitly models how normal behavior varies across contexts by learning conditional data distributions $P(\mathbf{Y} \mid \mathbf{C})$ rather than a global joint distribution $P(\mathbf{X})$. The framework encompasses (1) a probabilistic formulation for context-conditioned learning, (2) a principled bilevel optimization strategy for automatically selecting informative context features using early validation loss, and (3) theoretical grounding through variance decomposition and discriminative learning principles. We instantiate this framework using a novel conditional Wasserstein autoencoder as a simple yet effective model for tabular anomaly detection. Extensive experiments across eight benchmark datasets demonstrate that contextual learning consistently outperforms global approaches-even when the optimal context is not intuitively obvious-establishing a new foundation for anomaly detection in heterogeneous tabular data.

Method: Interactive Retrosynthesis Planning (InterRetro) that interacts with tree MDPs, learns value functions for worst-path outcomes, and improves policy through self-imitation with high-advantage decision reinforcement.

Result: Achieves 100% target solving on Retro*-190 benchmark, shortens synthetic routes by 4.9%, and shows strong performance with only 10% training data.

Conclusion: Worst-path optimization framework provides unique optimal solution with monotonic improvement guarantees, significantly advancing retrosynthesis planning robustness.

Abstract: Retrosynthesis planning aims to decompose target molecules into available building blocks, forming a synthetic tree where each internal node represents an intermediate compound and each leaf ideally corresponds to a purchasable reactant. However, this tree becomes invalid if any leaf node is not a valid building block, making the planning process vulnerable to the “weakest link” in the synthetic route. Existing methods often optimise for average performance across branches, failing to account for this worst-case sensitivity. In this paper, we reframe retrosynthesis as a worst-path optimisation problem within tree-structured Markov Decision Processes (MDPs). We prove that this formulation admits a unique optimal solution and provides monotonic improvement guarantees. Building on this insight, we introduce Interactive Retrosynthesis Planning (InterRetro), a method that interacts with the tree MDP, learns a value function for worst-path outcomes, and improves its policy through self-imitation, preferentially reinforcing past decisions with high estimated advantage. Empirically, InterRetro achieves state-of-the-art results - solving 100% of targets on the Retro*-190 benchmark, shortening synthetic routes by 4.9%, and achieving promising performance using only 10% of the training data.

Method: Proposes CAUSA with two components: 1) generalized unsupervised spectral regression to identify features via feature-clustering label dependencies, and 2) causal regularization module to separate confounders and learn view-shared sample weights to balance confounder distributions.

Result: Comprehensive experiments demonstrate that CAUSA outperforms several state-of-the-art methods in multi-view unsupervised feature selection.

Conclusion: This is the first in-depth study of causal multi-view feature selection in unsupervised settings, successfully mitigating spurious correlations to select causally informative features.

Abstract: Multi-view unsupervised feature selection (MUFS) has recently received increasing attention for its promising ability in dimensionality reduction on multi-view unlabeled data. Existing MUFS methods typically select discriminative features by capturing correlations between features and clustering labels. However, an important yet underexplored question remains: \textit{Are such correlations sufficiently reliable to guide feature selection?} In this paper, we analyze MUFS from a causal perspective by introducing a novel structural causal model, which reveals that existing methods may select irrelevant features because they overlook spurious correlations caused by confounders. Building on this causal perspective, we propose a novel MUFS method called CAusal multi-view Unsupervised feature Selection leArning (CAUSA). Specifically, we first employ a generalized unsupervised spectral regression model that identifies informative features by capturing dependencies between features and consensus clustering labels. We then introduce a causal regularization module that can adaptively separate confounders from multi-view data and simultaneously learn view-shared sample weights to balance confounder distributions, thereby mitigating spurious correlations. Thereafter, integrating both into a unified learning framework enables CAUSA to select causally informative features. Comprehensive experiments demonstrate that CAUSA outperforms several state-of-the-art methods. To our knowledge, this is the first in-depth study of causal multi-view feature selection in the unsupervised setting.

Method: Unified reimplementation of neural forecasting networks adapted for CFTL setup, pre-training only on proprietary and synthetic data while preventing test leakage, and evaluation on 15 large, diverse public forecast competition datasets.

Result: Statistical models and their ensembles consistently outperform existing foundation forecasting models by more than 8.2% in sCRPS and more than 20% in MASE across datasets. Synthetic dataset pre-training improves FFM accuracy by 7%.

Conclusion: Current CFTL benchmarking practices significantly underreport statistical models’ accuracy, and while statistical models outperform FFMs, synthetic pre-training does provide meaningful improvements to FFMs.

Abstract: Cross-frequency transfer learning (CFTL) has emerged as a popular framework for curating large-scale time series datasets to pre-train foundation forecasting models (FFMs). Although CFTL has shown promise, current benchmarking practices fall short of accurately assessing its performance. This shortcoming stems from many factors: an over-reliance on small-scale evaluation datasets; inadequate treatment of sample size when computing summary statistics; reporting of suboptimal statistical models; and failing to account for non-negligible risks of overlap between pre-training and test datasets. To address these limitations, we introduce a unified reimplementation of widely-adopted neural forecasting networks, adapting them for the CFTL setup; we pre-train only on proprietary and synthetic data, being careful to prevent test leakage; and we evaluate on 15 large, diverse public forecast competition datasets. Our empirical analysis reveals that statistical models’ accuracy is frequently underreported. Notably, we confirm that statistical models and their ensembles consistently outperform existing FFMs by more than 8.2% in sCRPS, and by more than 20% MASE, across datasets. However, we also find that synthetic dataset pre-training does improve the accuracy of a FFM by 7% percent.

Method: The chapter analyzes GeoML libraries’ architecture, data types, and ML framework integration, discusses preprocessing methodologies, spatial-temporal joins, benchmarking, and presents a crop type mapping case study.

Result: The analysis provides guidance on navigating the GeoML ecosystem, highlights best practices in software design, licensing, and testing, and identifies open challenges.

Conclusion: The chapter aims to guide practitioners, developers, and researchers in contributing to the rapidly evolving GeoML landscape, particularly addressing the rise of foundation models and governance needs in open-source geospatial software.

Abstract: Recent advances in machine learning have been supported by the emergence of domain-specific software libraries, enabling streamlined workflows and increased reproducibility. For geospatial machine learning (GeoML), the availability of Earth observation data has outpaced the development of domain libraries to handle its unique challenges, such as varying spatial resolutions, spectral properties, temporal cadence, data coverage, coordinate systems, and file formats. This chapter presents a comprehensive overview of GeoML libraries, analyzing their evolution, core functionalities, and the current ecosystem. It also introduces popular GeoML libraries such as TorchGeo, eo-learn, and Raster Vision, detailing their architecture, supported data types, and integration with ML frameworks. Additionally, it discusses common methodologies for data preprocessing, spatial–temporal joins, benchmarking, and the use of pretrained models. Through a case study in crop type mapping, it demonstrates practical applications of these tools. Best practices in software design, licensing, and testing are highlighted, along with open challenges and future directions, particularly the rise of foundation models and the need for governance in open-source geospatial software. Our aim is to guide practitioners, developers, and researchers in navigating and contributing to the rapidly evolving GeoML landscape.

Method: Hybrid backbone combining CNN feature extraction with transformer attention, using DDIM sampling for efficiency. Inputs include Reynolds number, angle of attack, and airfoil geometry encoding.

Result: 85% reduction in mean prediction errors compared to state-of-the-art models, with better accuracy and calibrated predictive uncertainty than existing diffusion models.

Conclusion: FoilDiff demonstrates superior performance in airfoil flow field prediction, offering significant improvements in accuracy and uncertainty calibration over current methods.

Abstract: The accurate prediction of flow fields around airfoils is crucial for aerodynamic design and optimisation. Computational Fluid Dynamics (CFD) models are effective but computationally expensive, thus inspiring the development of surrogate models to enable quicker predictions. These surrogate models can be based on deep learning architectures, such as Convolutional Neural Networks (CNNs), Graph Neural Networks (GNNs), and Diffusion Models (DMs). Diffusion models have shown significant promise in predicting complex flow fields. In this work, we propose FoilDiff, a diffusion-based surrogate model with a hybrid-backbone denoising network. This hybrid design combines the power of convolutional feature extraction and transformer-based global attention to generate more adaptable and accurate representations of flow structures. FoilDiff takes advantage of Denoising Diffusion Implicit Model (DDIM) sampling to optimise the efficiency of the sampling process at no additional cost to model generalisation. We used encoded representations of Reynolds number, angle of attack, and airfoil geometry to define the input space for generalisation across a wide range of aerodynamic conditions. When evaluated against state-of-the-art models, FoilDiff shows significant performance improvements, with mean prediction errors reducing by up to 85% on the same datasets. The results have demonstrated that FoilDiff can provide both more accurate predictions and better-calibrated predictive uncertainty than existing diffusion-based models.

Method: The authors combine diffusion models with an annealed variant of Langevin dynamics. This hybrid approach leverages the robustness of diffusion models to score estimation errors while maintaining the conditional sampling capabilities of Langevin dynamics.

Result: The proposed method achieves conditional sampling in polynomial time using only an L^4 bound on the score estimation error, which is significantly weaker than the sub-exponential bounds required by standard Langevin dynamics.

Conclusion: This work bridges the gap between unconditional diffusion models and conditional posterior sampling, providing a computationally efficient and robust framework for solving inverse problems with log-concave distributions.

Abstract: Given a noisy linear measurement $y = Ax + ξ$ of a distribution $p(x)$, and a good approximation to the prior $p(x)$, when can we sample from the posterior $p(x \mid y)$? Posterior sampling provides an accurate and fair framework for tasks such as inpainting, deblurring, and MRI reconstruction, and several heuristics attempt to approximate it. Unfortunately, approximate posterior sampling is computationally intractable in general. To sidestep this hardness, we focus on (local or global) log-concave distributions $p(x)$. In this regime, Langevin dynamics yields posterior samples when the exact scores of $p(x)$ are available, but it is brittle to score–estimation error, requiring an MGF bound (sub-exponential error). By contrast, in the unconditional setting, diffusion models succeed with only an $L^2$ bound on the score error. We prove that combining diffusion models with an annealed variant of Langevin dynamics achieves conditional sampling in polynomial time using merely an $L^4$ bound on the score error.

Method: Walsh-Assisted Relaxation for Probabilistic Look-Up Tables (WARP-LUTs) - a gradient-based approach that learns combinations of logic gates with substantially fewer trainable parameters.

Result: WARP-LUTs achieve significantly faster convergence on CIFAR-10 compared to DLGNs while maintaining comparable accuracy.

Conclusion: The approach shows potential for extension to higher-input logic blocks and efficient deployment on modern FPGAs for real-time science applications.

Abstract: Fast and efficient machine learning is of growing interest to the scientific community and has spurred significant research into novel model architectures and hardware-aware design. Recent hard? and software co-design approaches have demonstrated impressive results with entirely multiplication-free models. Differentiable Logic Gate Networks (DLGNs), for instance, provide a gradient-based framework for learning optimal combinations of low-level logic gates, setting state-of-the-art trade-offs between accuracy, resource usage, and latency. However, these models suffer from high computational cost during training and do not generalize well to logic blocks with more inputs. In this work, we introduce Walsh-Assisted Relaxation for Probabilistic Look-Up Tables (WARP-LUTs) - a novel gradient-based method that efficiently learns combinations of logic gates with substantially fewer trainable parameters. We demonstrate that WARP-LUTs achieve significantly faster convergence on CIFAR-10 compared to DLGNs, while maintaining comparable accuracy. Furthermore, our approach suggests potential for extension to higher-input logic blocks, motivating future research on extremely efficient deployment on modern FPGAs and its real-time science applications.

Method: GRWM applies temporal contrastive learning as geometric regularization to curate a latent space that better reflects the underlying physical state manifold. It’s a lightweight module that can be integrated into standard autoencoders to reshape latent space for stable dynamics modeling.

Result: The research demonstrates that high-fidelity cloning is feasible and identifies geometric structure of latent representation as the primary bottleneck for long-horizon fidelity, not the dynamics model itself. GRWM effectively improves world model fidelity through geometric regularization.

Conclusion: Contrastive constraints serve as a powerful inductive bias for stable world modeling. By focusing on representation quality, GRWM offers a simple yet powerful pipeline for improving world model fidelity in deterministic environments.

Abstract: A world model is an internal model that simulates how the world evolves. Given past observations and actions, it predicts the future physical state of both the embodied agent and its environment. Accurate world models are essential for enabling agents to think, plan, and reason effectively in complex, dynamic settings. However, existing world models often focus on random generation of open worlds, but neglect the need for high-fidelity modeling of deterministic scenarios (such as fixed-map mazes and static space robot navigation). In this work, we take a step toward building a truly accurate world model by addressing a fundamental yet open problem: constructing a model that can fully clone a deterministic 3D world. 1) Through diagnostic experiment, we quantitatively demonstrate that high-fidelity cloning is feasible and the primary bottleneck for long-horizon fidelity is the geometric structure of the latent representation, not the dynamics model itself. 2) Building on this insight, we show that applying temporal contrastive learning principle as a geometric regularization can effectively curate a latent space that better reflects the underlying physical state manifold, demonstrating that contrastive constraints can serve as a powerful inductive bias for stable world modeling; we call this approach Geometrically-Regularized World Models (GRWM). At its core is a lightweight geometric regularization module that can be seamlessly integrated into standard autoencoders, reshaping their latent space to provide a stable foundation for effective dynamics modeling. By focusing on representation quality, GRWM offers a simple yet powerful pipeline for improving world model fidelity.

Method: Investigates Markovian function learning through structured ICL setup, characterizes loss landscape, provides closed-form expression of global minimizer for single-layer linear self-attention, analyzes NP-hardness of parameter recovery, and interprets multilayer LSA as preconditioned gradient descent.

Result: Shows that recovering transformer parameters for optimal solution is NP-hard for single-layer LSA, revealing fundamental limitations in representing structured dynamical functions, and provides numerical validation using simplified transformers.

Conclusion: Single-layer linear self-attention has fundamental limitations in representing structured dynamical functions due to NP-hard parameter recovery, while multilayer architectures can be interpreted as performing preconditioned gradient descent for multiple objectives.

Abstract: Transformer architectures can solve unseen tasks based on input-output pairs in a given prompt due to in-context learning (ICL). Existing theoretical studies on ICL have mainly focused on linear regression tasks, often with i.i.d. inputs. To understand how transformers express ICL when modeling dynamics-driven functions, we investigate Markovian function learning through a structured ICL setup, where we characterize the loss landscape to reveal underlying optimization behaviors. Specifically, we (1) provide the closed-form expression of the global minimizer (in an enlarged parameter space) for a single-layer linear self-attention (LSA) model; (2) prove that recovering transformer parameters that realize the optimal solution is NP-hard in general, revealing a fundamental limitation of one-layer LSA in representing structured dynamical functions; and (3) supply a novel interpretation of a multilayer LSA as performing preconditioned gradient descent to optimize multiple objectives beyond the square loss. These theoretical results are numerically validated using simplified transformers.

Method: Integrates clustering with few-shot forecasting to identify site archetypes using a novel large-scale charging demand dataset, creating archetype-specific expert models for demand prediction.

Result: Archetype-specific expert models demonstrate superior forecasting performance compared to global baselines when predicting charging demand at previously unseen sites.

Conclusion: The framework enables actionable infrastructure segmentation insights that help operators reduce costs, optimize energy/pricing strategies, and support grid resilience essential for achieving climate goals.

Abstract: The decarbonization of transportation relies on the widespread adoption of electric vehicles (EVs), which requires an accurate understanding of charging behavior to ensure cost-effective, grid-resilient infrastructure. Existing work is constrained by small-scale datasets, simple proximity-based modeling of temporal dependencies, and weak generalization to sites with limited operational history. To overcome these limitations, this work proposes a framework that integrates clustering with few-shot forecasting to uncover site archetypes using a novel large-scale dataset of charging demand. The results demonstrate that archetype-specific expert models outperform global baselines in forecasting demand at unseen sites. By establishing forecast performance as a basis for infrastructure segmentation, we generate actionable insights that enable operators to lower costs, optimize energy and pricing strategies, and support grid resilience critical to climate goals.

Method: Physics-Informed Neural Networks (PINNs) with embedded physical laws, systematic hyperparameter tuning, loss function balancing (data, initial condition, residual losses), and hard constraint imposition.

Result: PINNs achieve superior results for complex ODE problems when loss components are properly balanced and hyperparameters are systematically tuned, demonstrating enhanced predictive capability.

Conclusion: PINNs provide a powerful framework for solving challenging ODE problems, though they require careful implementation including loss balancing, hyperparameter optimization, and constraint embedding for optimal performance.

Abstract: In this study, we present and validate the predictive capability of the Physics-Informed Neural Networks (PINNs) methodology for solving a variety of engineering and biological dynamical systems governed by ordinary differential equations (ODEs). While traditional numerical methods a re effective for many ODEs, they often struggle to achieve convergence in problems involving high stiffness, shocks, irregular domains, singular perturbations, high dimensions, or boundary discontinuities. Alternatively, PINNs offer a powerful approach for handling challenging numerical scenarios. In this study, classical ODE problems are employed as controlled testbeds to systematically evaluate the accuracy, training efficiency, and generalization capability under controlled conditions of the PINNs framework. Although not a universal solution, PINNs can achieve superior results by embedding physical laws directly into the learning process. We first analyze the existence and uniqueness properties of several benchmark problems and subsequently validate the PINNs methodology on these model systems. Our results demonstrate that for complex problems to converge to correct solutions, the loss function components data loss, initial condition loss, and residual loss must be appropriately balanced through careful weighting. We further establish that systematic tuning of hyperparameters, including network depth, layer width, activation functions, learning rate, optimization algorithms, w eight initialization schemes, and collocation point sampling, plays a crucial role in achieving accurate solutions. Additionally, embedding prior knowledge and imposing hard constraints on the network architecture, without loss the generality of the ODE system, significantly enhances the predictive capability of PINNs.

Method: Developed PINN integrating physical laws (mass conservation, Fick’s diffusion, Henry’s solubility) with compact neural network architecture (17,793 parameters) validated across various membranes and operating conditions.

Result: Achieved exceptional accuracy (R² = 99.84% ± 0.15%, RMSE = 0.0932% ± 0.0438%) with sub-millisecond inference times, maintaining R² > 86% when predicting beyond training range.

Conclusion: PINN bridges physical rigor and computational efficiency, enabling real-time electrolyzer monitoring essential for safe, large-scale green hydrogen infrastructure deployment.

Abstract: Green hydrogen production via polymer electrolyte membrane (PEM) water electrolysis is pivotal for energy transition, yet hydrogen crossover through membranes threatens safety and economic viability-approaching explosive limits (4 mol% H$_2$ in O$_2$) while reducing Faradaic efficiency by 2.5%. Current physics-based models require extensive calibration and computational resources that preclude real-time implementation, while purely data-driven approaches fail to extrapolate beyond training conditions-critical for dynamic electrolyzer operation. Here we present the first application of physics-informed neural networks (PINNs) for hydrogen crossover prediction, integrating mass conservation, Fick’s diffusion law, and Henry’s solubility law within a compact architecture (17,793 parameters). Validated across six membranes under industrially relevant conditions (0.05-5.0 A/cm$^2$, 1-200 bar, 25-85°C), our PINN achieves exceptional accuracy (R$^{2}$ = 99.84% $\pm$ 0.15%, RMSE = 0.0932% $\pm$ 0.0438%) based on five-fold cross-validation, with sub-millisecond inference times suitable for real-time control. Remarkably, the model maintains R$^2$ > 86% when predicting crossover at pressures 2.5x beyond training range-substantially outperforming pure neural networks (R$^2$ = 43.4%). The hardware-agnostic deployment, from desktop CPUs to edge devices (Raspberry Pi 4), enables distributed safety monitoring essential for gigawatt-scale installations. By bridging physical rigor and computational efficiency, this work establishes a new paradigm for real-time electrolyzer monitoring, accelerating deployment of safe, efficient green hydrogen infrastructure crucial for net-zero emissions targets.

Method: Proposes ReLaX-Net architecture using layer-by-layer time-multiplexing to increase effective network depth. Requires only fast switches added to existing PNNs, leveraging time-scale separation between fast forward pass dynamics and slowly trainable weight elements.

Result: Numerical experiments on image classification and NLP tasks show ReLaX-Net improves computational performance with minor PNN modifications. Exhibits favorable scaling, outperforming equivalent traditional RNNs/DNNs with same parameter count.

Conclusion: ReLaX-Net enables efficient parameter reuse in PNNs through time-multiplexing, addressing scale limitations and demonstrating improved performance with minimal hardware overhead, making PNNs more competitive with digital counterparts.

Abstract: Physical Neural Networks (PNN) are promising platforms for next-generation computing systems. However, recent advances in digital neural network performance are largely driven by the rapid growth in the number of trainable parameters and, so far, demonstrated PNNs are lagging behind by several orders of magnitude in terms of scale. This mirrors size and performance constraints found in early digital neural networks. In that period, efficient reuse of parameters contributed to the development of parameter-efficient architectures such as convolutional neural networks. In this work, we numerically investigate hardware-friendly weight-tying for PNNs. Crucially, with many PNN systems, there is a time-scale separation between the fast dynamic active elements of the forward pass and the only slowly trainable elements implementing weights and biases. With this in mind,we propose the Reuse of Layers for eXpanding a Neural Network (ReLaX-Net) architecture, which employs a simple layer-by-layer time-multiplexing scheme to increase the effective network depth and efficiently use the number of parameters. We only require the addition of fast switches for existing PNNs. We validate ReLaX-Nets via numerical experiments on image classification and natural language processing tasks. Our results show that ReLaX-Net improves computational performance with only minor modifications to a conventional PNN. We observe a favorable scaling, where ReLaX-Nets exceed the performance of equivalent traditional RNNs or DNNs with the same number of parameters.

Method: Multi-level masking strategy for partial observations + hierarchical prompt learning that aggregates unmasked data into global prompts to guide feature modeling.

Result: State-of-the-art performance on four datasets, especially effective with very limited data amounts.

Conclusion: MASS enables efficient deployment in real-world low-resource sleep monitoring environments by reducing signal requirements while maintaining reliability.

Abstract: Automatic sleep staging plays a vital role in assessing sleep quality and diagnosing sleep disorders. Most existing methods rely heavily on long and continuous EEG recordings, which poses significant challenges for data acquisition in resource-constrained systems, such as wearable or home-based monitoring systems. In this paper, we propose the task of resource-efficient sleep staging, which aims to reduce the amount of signal collected per sleep epoch while maintaining reliable classification performance. To solve this task, we adopt the masking and prompt learning strategy and propose a novel framework called Mask-Aware Sleep Staging (MASS). Specifically, we design a multi-level masking strategy to promote effective feature modeling under partial and irregular observations. To mitigate the loss of contextual information introduced by masking, we further propose a hierarchical prompt learning mechanism that aggregates unmasked data into a global prompt, serving as a semantic anchor for guiding both patch-level and epoch-level feature modeling. MASS is evaluated on four datasets, demonstrating state-of-the-art performance, especially when the amount of data is very limited. This result highlights its potential for efficient and scalable deployment in real-world low-resource sleep monitoring environments.

Method: Two-stage approach: 1) Rank residual blocks by label-aware mutual information and prune least informative units, 2) Alternate knowledge distillation with stage-coherent channel slicing (stage planes and optional mid-channel trim) while preserving residual shape invariants.

Result: Achieved significant compression: ResNet-18 (72% params, 63% GMacs reduction), ResNet-34 (74% params/GMacs reduction), MobileNetV2 (73% params, 76% conv GMacs reduction) while maintaining or improving accuracy (e.g., MobileNetV2 improved by +2.5 points).

Conclusion: M2M-DC provides a practical, architecture-aware compression framework that generalizes across residual CNNs and inverted-residual families, delivering compact models ready for deployment with comparable or better accuracy at reduced computational cost.

Abstract: We introduce MI-to-Mid Distilled Compression (M2M-DC), a two-scale, shape-safe compression framework that interleaves information-guided block pruning with progressive inner slicing and staged knowledge distillation (KD). First, M2M-DC ranks residual (or inverted-residual) blocks by a label-aware mutual information (MI) signal and removes the least informative units (structured prune-after-training). It then alternates short KD phases with stage-coherent, residual-safe channel slicing: (i) stage “planes” (co-slicing conv2 out-channels with the downsample path and next-stage inputs), and (ii) an optional mid-channel trim (conv1 out / bn1 / conv2 in). This targets complementary redundancy, whole computational motifs and within-stage width while preserving residual shape invariants. On CIFAR-100, M2M-DC yields a clean accuracy-compute frontier. For ResNet-18, we obtain 85.46% Top-1 with 3.09M parameters and 0.0139 GMacs (72% params, 63% GMacs vs. teacher; mean final 85.29% over three seeds). For ResNet-34, we reach 85.02% Top-1 with 5.46M params and 0.0195 GMacs (74% / 74% vs. teacher; mean final 84.62%). Extending to inverted-residuals, MobileNetV2 achieves a mean final 68.54% Top-1 at 1.71M params (27%) and 0.0186 conv GMacs (24%), improving over the teacher’s 66.03% by +2.5 points across three seeds. Because M2M-DC exposes only a thin, architecture-aware interface (blocks, stages, and down sample/skip wiring), it generalizes across residual CNNs and extends to inverted-residual families with minor legalization rules. The result is a compact, practical recipe for deployment-ready models that match or surpass teacher accuracy at a fraction of the compute.

Method: Proposes Self-Examining RL (SERL) with two internal reward mechanisms: (1) Copeland-style pairwise comparison judgments across generated responses for Actor improvement, (2) self-consistency reward for Judge reliability improvement.

Result: Outperforms existing self-improvement methods, improves Qwen3-8B LC win rate on AlpacaEval 2 from 52.37% to 59.90%, achieves SOTA among self-improving approaches, and matches performance of much larger models like Qwen3-32B.

Conclusion: SERL provides an effective self-improving framework that eliminates need for external rewards while achieving superior performance and robustness on open-domain tasks.

Abstract: Reinforcement Learning (RL) has been shown to improve the capabilities of large language models (LLMs). However, applying RL to open-domain tasks faces two key challenges: (1) the inherent subjectivity of these tasks prevents the verifiable rewards as required by Reinforcement Learning with Verifiable Rewards (RLVR); (2) Reinforcement Learning from Human Feedback (RLHF) relies on external reward mechanisms. To overcome these limitations, we propose Self-Examining Reinforcement Learning (SERL), a novel self-improving framework where the LLM serves as both Actor and Judge. SERL introduces two synergistic reward mechanisms without any external signals. On the one hand, to improve the Actor’s capability, we derive rewards from Copeland-style pairwise comparison judgments across a group of generated responses. On the other hand, a self-consistency reward that encourages coherent judgments is proposed to improve the Judge’s reliability. This process refines the Judge’s capability, which in turn provides a more robust reward for Actor. Experiments show that our method outperforms existing self-improvement training methods. SERL improves the LC win rate of Qwen3-8B on AlpacaEval 2 from 52.37% to 59.90%. To the best of our knowledge, our method achieves state-of-the-art performance among self-improving approaches. Furthermore, it achieves a performance comparable to significantly larger models like Qwen3-32B, demonstrating superior effectiveness and robustness on open-domain tasks.

Method: Extends Matryoshka representation learning to transcoder architectures for hierarchical sparse feature learning at multiple granularity levels, training on intermediate representations from physical plausibility classifiers and using large multimodal models for interpretation.

Result: Achieves superior feature relevance and accuracy compared to existing approaches, identifies diverse physics-related failure modes without manual feature engineering, and provides a benchmark for evaluating physical plausibility in generative models.

Conclusion: Analysis of eight state-of-the-art generative models reveals valuable insights into how these models fail to follow physical constraints, paving the way for further model improvements.

Abstract: Although recent generative models are remarkably capable of producing instruction-following and realistic outputs, they remain prone to notable physical plausibility failures. Though critical in applications, these physical plausibility errors often escape detection by existing evaluation methods. Furthermore, no framework exists for automatically identifying and interpreting specific physical error patterns in natural language, preventing targeted model improvements. We introduce Matryoshka Transcoders, a novel framework for the automatic discovery and interpretation of physical plausibility features in generative models. Our approach extends the Matryoshka representation learning paradigm to transcoder architectures, enabling hierarchical sparse feature learning at multiple granularity levels. By training on intermediate representations from a physical plausibility classifier and leveraging large multimodal models for interpretation, our method identifies diverse physics-related failure modes without manual feature engineering, achieving superior feature relevance and feature accuracy compared to existing approaches. We utilize the discovered visual patterns to establish a benchmark for evaluating physical plausibility in generative models. Our analysis of eight state-of-the-art generative models provides valuable insights into how these models fail to follow physical constraints, paving the way for further model improvements.

Method: ScaPT uses a meta-agent for efficient parameter sharing across agents and a mutual information regularizer to maintain population diversity.

Result: ScaPT shows superior performance compared to representative frameworks in the Hanabi cooperative game.

Conclusion: The proposed ScaPT framework effectively addresses scalability limitations in zero-shot coordination while maintaining performance through efficient parameter sharing and diversity preservation.

Abstract: Zero-shot coordination(ZSC), a key challenge in multi-agent game theory, has become a hot topic in reinforcement learning (RL) research recently, especially in complex evolving games. It focuses on the generalization ability of agents, requiring them to coordinate well with collaborators from a diverse, potentially evolving, pool of partners that are not seen before without any fine-tuning. Population-based training, which approximates such an evolving partner pool, has been proven to provide good zero-shot coordination performance; nevertheless, existing methods are limited by computational resources, mainly focusing on optimizing diversity in small populations while neglecting the potential performance gains from scaling population size. To address this issue, this paper proposes the Scalable Population Training (ScaPT), an efficient RL training framework comprising two key components: a meta-agent that efficiently realizes a population by selectively sharing parameters across agents, and a mutual information regularizer that guarantees population diversity. To empirically validate the effectiveness of ScaPT, this paper evaluates it along with representational frameworks in Hanabi cooperative game and confirms its superiority.

Method: Builds on SamAdams framework, using time rescaling to modulate stepsize according to monitored gradient norms, automatically shrinking stepsizes in high-curvature regions and expanding in flatter regions.

Result: Achieves more accurate posterior sampling than SGLD on high-curvature 2D toy examples and in image classification with BNNs using sharp priors.

Conclusion: SA-SGLD provides improved stability and mixing for Bayesian neural network posterior sampling without introducing bias, outperforming standard SGLD in challenging high-curvature scenarios.

Abstract: Bayesian neural networks (BNNs) require scalable sampling algorithms to approximate posterior distributions over parameters. Existing stochastic gradient Markov Chain Monte Carlo (SGMCMC) methods are highly sensitive to the choice of stepsize and adaptive variants such as pSGLD typically fail to sample the correct invariant measure without addition of a costly divergence correction term. In this work, we build on the recently proposed `SamAdams’ framework for timestep adaptation (Leimkuhler, Lohmann, and Whalley 2025), introducing an adaptive scheme: SA-SGLD, which employs time rescaling to modulate the stepsize according to a monitored quantity (typically the local gradient norm). SA-SGLD can automatically shrink stepsizes in regions of high curvature and expand them in flatter regions, improving both stability and mixing without introducing bias. We show that our method can achieve more accurate posterior sampling than SGLD on high-curvature 2D toy examples and in image classification with BNNs using sharp priors.

Method: Developed a Bayesian model for funnel decision structures, building on prior work on selective labels and censoring, and validated it using synthetic settings and emergency department visit data.

Result: The model accurately recovered true parameters and predicted outcomes for censored patients better than baselines. Applied to emergency department data, it revealed gender-based admission differences: women require higher mortality risk thresholds (5.1%) than men (4.5%) for ICU admission.

Conclusion: The Bayesian model effectively addresses selective censoring bias in healthcare funnel structures and uncovers disparities in decision thresholds across patient groups.

Abstract: Many sequential decision settings in healthcare feature funnel structures characterized by a series of stages, such as screenings or evaluations, where the number of patients who advance to each stage progressively decreases and decisions become increasingly costly. For example, an oncologist may first conduct a breast exam, followed by a mammogram for patients with concerning exams, followed by a biopsy for patients with concerning mammograms. A key challenge is that the ground truth outcome, such as the biopsy result, is only revealed at the end of this funnel. The selective censoring of the ground truth can introduce statistical biases in risk estimation, especially in underserved patient groups, whose outcomes are more frequently censored. We develop a Bayesian model for funnel decision structures, drawing from prior work on selective labels and censoring. We first show in synthetic settings that our model is able to recover the true parameters and predict outcomes for censored patients more accurately than baselines. We then apply our model to a dataset of emergency department visits, where in-hospital mortality is observed only for those who are admitted to either the hospital or ICU. We find that there are gender-based differences in hospital and ICU admissions. In particular, our model estimates that the mortality risk threshold to admit women to the ICU is higher for women (5.1%) than for men (4.5%).

Method: Two-stage representation learning: first model sensors independently, then combine representations into shared model at native 10m resolution and cloud-free Sentinel-2 frequency.

Result: Learned embeddings show high spatial/semantic consistency across landscapes and encode ecologically meaningful patterns for Gross Primary Production modeling.

Conclusion: The framework provides flexible, analysis-ready representation learning for environmental applications requiring diverse spatial and temporal resolutions.

Abstract: Earth observation (EO) foundation models have emerged as an effective approach to derive latent representations of the Earth system from various remote sensing sensors. These models produce embeddings that can be used as analysis-ready datasets, enabling the modelling of ecosystem dynamics without extensive sensor-specific preprocessing. However, existing models typically operate at fixed spatial or temporal scales, limiting their use for ecological analyses that require both fine spatial detail and high temporal fidelity. To overcome these limitations, we propose a representation learning framework that integrates different EO modalities into a unified feature space at high spatio-temporal resolution. We introduce the framework using Sentinel-1 and Sentinel-2 data as representative modalities. Our approach produces a latent space at native 10 m resolution and the temporal frequency of cloud-free Sentinel-2 acquisitions. Each sensor is first modeled independently to capture its sensor-specific characteristics. Their representations are then combined into a shared model. This two-stage design enables modality-specific optimisation and easy extension to new sensors, retaining pretrained encoders while retraining only fusion layers. This enables the model to capture complementary remote sensing data and to preserve coherence across space and time. Qualitative analyses reveal that the learned embeddings exhibit high spatial and semantic consistency across heterogeneous landscapes. Quantitative evaluation in modelling Gross Primary Production reveals that they encode ecologically meaningful patterns and retain sufficient temporal fidelity to support fine-scale analyses. Overall, the proposed framework provides a flexible, analysis-ready representation learning approach for environmental applications requiring diverse spatial and temporal resolutions.

Method: Integrates KANs within fair adversarial learning framework, uses spline-based KAN architecture for stability, and adaptive fairness penalty update mechanism.

Result: Empirical evidence on real-world admissions datasets shows efficient fairness achievement across sensitive attributes while preserving predictive performance.

Conclusion: The proposed framework successfully addresses fairness-accuracy trade-off and interpretability challenges in sensitive decision-making domains.

Abstract: Despite recent advances in fairness-aware machine learning, predictive models often exhibit discriminatory behavior towards marginalized groups. Such unfairness might arise from biased training data, model design, or representational disparities across groups, posing significant challenges in high-stakes decision-making domains such as college admissions. While existing fair learning models aim to mitigate bias, achieving an optimal trade-off between fairness and accuracy remains a challenge. Moreover, the reliance on black-box models hinders interpretability, limiting their applicability in socially sensitive domains. To circumvent these issues, we propose integrating Kolmogorov-Arnold Networks (KANs) within a fair adversarial learning framework. Leveraging the adversarial robustness and interpretability of KANs, our approach facilitates stable adversarial learning. We derive theoretical insights into the spline-based KAN architecture that ensure stability during adversarial optimization. Additionally, an adaptive fairness penalty update mechanism is proposed to strike a balance between fairness and accuracy. We back these findings with empirical evidence on two real-world admissions datasets, demonstrating the proposed framework’s efficiency in achieving fairness across sensitive attributes while preserving predictive performance.

Method: Proposed MPD-SGR (Membrane Potential Distribution-driven Surrogate Gradient Regularization) that explicitly regularizes the membrane potential distribution based on its interaction with the surrogate gradient function to reduce the proportion of membrane potentials within the gradient-available range.

Result: Extensive experiments across multiple image classification benchmarks and diverse network architectures show MPD-SGR significantly enhances SNN resilience to adversarial perturbations and exhibits strong generalizability across different network configurations, SG functions, and spike encoding schemes.

Conclusion: Reducing the proportion of membrane potentials within the gradient-available range of surrogate gradient functions effectively mitigates SNN sensitivity to input perturbations, and the proposed MPD-SGR method successfully enhances robustness through explicit membrane potential distribution regularization.

Abstract: The surrogate gradient (SG) method has shown significant promise in enhancing the performance of deep spiking neural networks (SNNs), but it also introduces vulnerabilities to adversarial attacks. Although spike coding strategies and neural dynamics parameters have been extensively studied for their impact on robustness, the critical role of gradient magnitude, which reflects the model’s sensitivity to input perturbations, remains underexplored. In SNNs, the gradient magnitude is primarily determined by the interaction between the membrane potential distribution (MPD) and the SG function. In this study, we investigate the relationship between the MPD and SG and their implications for improving the robustness of SNNs. Our theoretical analysis reveals that reducing the proportion of membrane potentials lying within the gradient-available range of the SG function effectively mitigates the sensitivity of SNNs to input perturbations. Building upon this insight, we propose a novel MPD-driven surrogate gradient regularization (MPD-SGR) method, which enhances robustness by explicitly regularizing the MPD based on its interaction with the SG function. Extensive experiments across multiple image classification benchmarks and diverse network architectures confirm that the MPD-SGR method significantly enhances the resilience of SNNs to adversarial perturbations and exhibits strong generalizability across diverse network configurations, SG functions, and spike encoding schemes.

Method: Novel checkpoint sparsification and quantization method that adapts dynamically to different training stages and model architectures. Uses bitmask-based sparsification and cluster-based quantization techniques.

Result: Bitmask-based sparsification achieves 16x compression ratio without compromising model accuracy. Cluster-based quantization achieves 2x compression ratio with little precision loss.

Conclusion: The proposed adaptive approach effectively balances compression ratio, speed, and precision impact throughout the training process, addressing current limitations in LLM checkpoint management.

Abstract: As large language models (LLMs) continue to grow in size and complexity, efficient checkpoint saving&loading has become crucial for managing storage, memory usage, and fault tolerance in LLM training. The current works do not comprehensively take into account the optimization of these several aspects. This paper proposes a novel checkpoint sparsification and quantization method that adapts dynamically to different training stages and model architectures. We present a comprehensive analysis of existing lossy and lossless compression techniques, identify current limitations, and introduce our adaptive approach that balances compression ratio, speed, and precision impact throughout the training process. Experiments on different sizes of LLMs demonstrate that our bitmask-based sparsification method achieves 16x compression ratio without compromising model accuracy. Additionally, the cluster-based quantization method achieves 2x compression ratio with little precision loss.

Method: Introduces synthetic data generator capturing temporal structures and client heterogeneity, maps time series to intrinsic representation space, and decomposes forecasting loss into Bayesian (irreducible uncertainty) and approximation (finite-sample effects) terms.

Result: Analysis shows increasing look-back horizon improves deterministic pattern identifiability but increases approximation error due to higher model complexity and reduced sample efficiency. Total loss minimized at smallest horizon where irreducible loss saturates while approximation loss rises.

Conclusion: Provides rigorous theoretical foundation for adaptive horizon selection in federated time series forecasting, establishing optimal horizon selection criteria based on loss decomposition.

Abstract: Selecting an appropriate look-back horizon remains a fundamental challenge in time series forecasting (TSF), particularly in the federated learning scenarios where data is decentralized, heterogeneous, and often non-independent. While recent work has explored horizon selection by preserving forecasting-relevant information in an intrinsic space, these approaches are primarily restricted to centralized and independently distributed settings. This paper presents a principled framework for adaptive horizon selection in federated time series forecasting through an intrinsic space formulation. We introduce a synthetic data generator (SDG) that captures essential temporal structures in client data, including autoregressive dependencies, seasonality, and trend, while incorporating client-specific heterogeneity. Building on this model, we define a transformation that maps time series windows into an intrinsic representation space with well-defined geometric and statistical properties. We then derive a decomposition of the forecasting loss into a Bayesian term, which reflects irreducible uncertainty, and an approximation term, which accounts for finite-sample effects and limited model capacity. Our analysis shows that while increasing the look-back horizon improves the identifiability of deterministic patterns, it also increases approximation error due to higher model complexity and reduced sample efficiency. We prove that the total forecasting loss is minimized at the smallest horizon where the irreducible loss starts to saturate, while the approximation loss continues to rise. This work provides a rigorous theoretical foundation for adaptive horizon selection for time series forecasting in federated learning.

Method: Propose Indirect Neural Corrector (INC) which integrates learned corrections into the governing equations rather than applying direct state updates, reducing error amplification on the order of Δt⁻¹ + L (timestep and Lipschitz constant), with no architectural requirements and seamless integration with arbitrary neural networks and solvers.

Result: INC improves long-term trajectory performance (R²) by up to 158.7%, stabilizes blowups under aggressive coarsening, and for complex 3D turbulence cases yields speed-ups of several orders of magnitude across extensive benchmarks covering numerous differentiable solvers, neural backbones, and test cases from 1D chaotic systems to 3D turbulence.

Conclusion: INC enables stable, efficient PDE emulation with formal error reduction, paving the way for faster scientific and engineering simulations with reliable physics guarantees.

Abstract: When simulating partial differential equations, hybrid solvers combine coarse numerical solvers with learned correctors. They promise accelerated simulations while adhering to physical constraints. However, as shown in our theoretical framework, directly applying learned corrections to solver outputs leads to significant autoregressive errors, which originate from amplified perturbations that accumulate during long-term rollouts, especially in chaotic regimes. To overcome this, we propose the Indirect Neural Corrector ($\mathrm{INC}$), which integrates learned corrections into the governing equations rather than applying direct state updates. Our key insight is that $\mathrm{INC}$ reduces the error amplification on the order of $Δt^{-1} + L$, where $Δt$ is the timestep and $L$ the Lipschitz constant. At the same time, our framework poses no architectural requirements and integrates seamlessly with arbitrary neural networks and solvers. We test $\mathrm{INC}$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence. $\mathrm{INC}$ improves the long-term trajectory performance ($R^2$) by up to 158.7%, stabilizes blowups under aggressive coarsening, and for complex 3D turbulence cases yields speed-ups of several orders of magnitude. $\mathrm{INC}$ thus enables stable, efficient PDE emulation with formal error reduction, paving the way for faster scientific and engineering simulations with reliable physics guarantees. Our source code is available at https://github.com/tum-pbs/INC

Method: Conducted geometric analysis of the network’s energy landscape using a novel ‘Pinnacle Sharpness’ metric, systematically varied kernel width and storage load, and theoretically decomposed the landscape gradient into direct driving and indirect feedback forces.

Result: Uncovered a rich phase diagram of attractor shapes and identified a ‘ridge of optimization’ where attractor stability is maximized under high-load conditions through strong anti-correlation between driving and feedback forces.

Conclusion: The network adaptively harnesses inter-pattern interactions as a cooperative feedback control system to sculpt a robust energy landscape, providing new physical insights for designing high-capacity associative memories.

Abstract: Kernel-based learning methods can dramatically increase the storage capacity of Hopfield networks, yet the dynamical mechanism behind this enhancement remains poorly understood. We address this gap by conducting a geometric analysis of the network’s energy landscape. We introduce a novel metric, “Pinnacle Sharpness,” to quantify the local stability of attractors. By systematically varying the kernel width and storage load, we uncover a rich phase diagram of attractor shapes. Our central finding is the emergence of a “ridge of optimization,” where the network maximizes attractor stability under challenging high-load and global-kernel conditions. Through a theoretical decomposition of the landscape gradient into a direct “driving” force and an indirect “feedback” force, we reveal the origin of this phenomenon. The optimization ridge corresponds to a regime of strong anti-correlation between the two forces, where the direct force, amplified by the high storage load, dominates the opposing collective feedback force. This demonstrates a sophisticated self-organization mechanism: the network adaptively harnesses inter-pattern interactions as a cooperative feedback control system to sculpt a robust energy landscape. Our findings provide a new physical picture for the stability of high-capacity associative memories and offer principles for their design.

Method: Uses partial demographic data to generate demographic proxies, enforces consistent embeddings across groups, and employs adaptive confidence strategy that dynamically adjusts node contributions based on prediction confidence.

Result: Theoretical analysis shows provable upper bounds on group fairness metrics. Extensive experiments demonstrate effectiveness in bias mitigation while maintaining model utility across multiple datasets.

Conclusion: FairGLite provides a practical solution for fair graph learning with limited demographic information, offering formal guarantees for bias mitigation without compromising utility.

Abstract: Ensuring fairness in Graph Neural Networks is fundamental to promoting trustworthy and socially responsible machine learning systems. In response, numerous fair graph learning methods have been proposed in recent years. However, most of them assume full access to demographic information, a requirement rarely met in practice due to privacy, legal, or regulatory restrictions. To this end, this paper introduces a novel fair graph learning framework that mitigates bias in graph learning under limited demographic information. Specifically, we propose a mechanism guided by partial demographic data to generate proxies for demographic information and design a strategy that enforces consistent node embeddings across demographic groups. In addition, we develop an adaptive confidence strategy that dynamically adjusts each node’s contribution to fairness and utility based on prediction confidence. We further provide theoretical analysis demonstrating that our framework, FairGLite, achieves provable upper bounds on group fairness metrics, offering formal guarantees for bias mitigation. Through extensive experiments on multiple datasets and fair graph learning frameworks, we demonstrate the framework’s effectiveness in both mitigating bias and maintaining model utility.

Method: Proposed decoupling quasi-Newton updates using coroutines while maintaining batched acquisition function calls, enabling theoretically identical convergence to sequential multi-start optimization but with reduced wall-clock time.

Result: The approach drastically reduces computational overhead compared to previous methods while maintaining theoretical convergence guarantees, and has been implemented in GPSampler within the Optuna library.

Conclusion: The proposed method effectively addresses the computational bottleneck in Bayesian optimization by combining the efficiency of batching with proper quasi-Newton update decoupling, providing significant speed improvements without compromising optimization quality.

Abstract: Bayesian optimization (BO) efficiently finds high-performing parameters by maximizing an acquisition function, which models the promise of parameters. A major computational bottleneck arises in acquisition function optimization, where multi-start optimization (MSO) with quasi-Newton (QN) methods is required due to the non-convexity of the acquisition function. BoTorch, a widely used BO library, currently optimizes the summed acquisition function over multiple points, leading to the speedup of MSO owing to PyTorch batching. Nevertheless, this paper empirically demonstrates the suboptimality of this approach in terms of off-diagonal approximation errors in the inverse Hessian of a QN method, slowing down its convergence. To address this problem, we propose to decouple QN updates using a coroutine while batching the acquisition function calls. Our approach not only yields the theoretically identical convergence to the sequential MSO but also drastically reduces the wall-clock time compared to the previous approaches. Our approach is available in GPSampler in Optuna, effectively reducing its computational overhead.

Method: Proposed FairSkillMARL framework with dual fairness objectives (workload balance + skill-task alignment) and created MARLHospital environment for testing team compositions and energy-constrained scheduling impacts.

Result: Experiments showed that fairness based solely on equal workload leads to task-skill mismatches, highlighting the need for more robust metrics that capture skill-task alignment.

Conclusion: The work provides tools and foundation for studying fairness in heterogeneous multi-agent systems where aligning effort with expertise is critical, moving beyond simple workload balancing.

Abstract: Fairness in multi-agent reinforcement learning (MARL) is often framed as a workload balance problem, overlooking agent expertise and the structured coordination required in real-world domains. In healthcare, equitable task allocation requires workload balance or expertise alignment to prevent burnout and overuse of highly skilled agents. Workload balance refers to distributing an approximately equal number of subtasks or equalised effort across healthcare workers, regardless of their expertise. We make two contributions to address this problem. First, we propose FairSkillMARL, a framework that defines fairness as the dual objective of workload balance and skill-task alignment. Second, we introduce MARLHospital, a customizable healthcare-inspired environment for modeling team compositions and energy-constrained scheduling impacts on fairness, as no existing simulators are well-suited for this problem. We conducted experiments to compare FairSkillMARL in conjunction with four standard MARL methods, and against two state-of-the-art fairness metrics. Our results suggest that fairness based solely on equal workload might lead to task-skill mismatches and highlight the need for more robust metrics that capture skill-task misalignment. Our work provides tools and a foundation for studying fairness in heterogeneous multi-agent systems where aligning effort with expertise is critical.

Method: Apply automatic differentiation (AD) to ABMs to make gradients available, then use variational inference (VI) techniques for parameter calibration on three ABMs: Axtell’s model of firms, Sugarscape, and the SIR epidemiological model.

Result: Experiments show substantial performance improvements and computational savings using VI with AD-enabled ABMs.

Conclusion: The approach significantly enhances the practicality and scalability of ABMs for studying complex systems by reducing computational burdens.

Abstract: Agent-based models (ABMs) simulate complex systems by capturing the bottom-up interactions of individual agents comprising the system. Many complex systems of interest, such as epidemics or financial markets, involve thousands or even millions of agents. Consequently, ABMs often become computationally demanding and rely on the calibration of numerous free parameters, which has significantly hindered their widespread adoption. In this paper, we demonstrate that automatic differentiation (AD) techniques can effectively alleviate these computational burdens. By applying AD to ABMs, the gradients of the simulator become readily available, greatly facilitating essential tasks such as calibration and sensitivity analysis. Specifically, we show how AD enables variational inference (VI) techniques for efficient parameter calibration. Our experiments demonstrate substantial performance improvements and computational savings using VI on three prominent ABMs: Axtell’s model of firms; Sugarscape; and the SIR epidemiological model. Our approach thus significantly enhances the practicality and scalability of ABMs for studying complex systems.

Method: Spatially explicit agent-based model integrating road-network GIS, synthetic populations from Iterative Proportional Fitting, and behavioral heterogeneity, applied to Premeno, Italy with baseline vs relocation scenarios analyzed across 40 batches and 50 replications.

Result: Aggregate neutrality but pronounced local redistribution of accessibility; spatial impedance dominates accessibility while behavioral capacity modulates care effort.

Conclusion: Demonstrates complex adaptive social system properties (emergence, heterogeneity, feedback) and how computational simulation can illuminate policy trade-offs between spatial efficiency, social equity, and care sustainability.

Abstract: Ageing societies face increasing strain on formal and informal care systems, par- ticularly in low-density mountainous municipalities where sparse services and steep terrain constrain access. This study presents a spatially explicit agent-based model that integrates a road-network GIS, synthetic populations derived through Iterative Proportional Fitting, and behavioural heterogeneity to examine how alternative service configurations shape accessibility and caregiver burden. The model, applied to Premeno (Piedmont, Italy), compares a baseline distribution of ambulatory services with a relocation scenario at Villa Bernocchi. System-level indicators (Caregiver Effort, Overwhelm, Hours Not Cared, Walkability) and micro-spatial metrics (Walkability, Detour Ratio, Proximity) are analysed across 40 batches and 50 stochastic replications per scenario. Results reveal aggregate neutrality but pronounced local redistribution of accessibility. Sensitivity analysis shows that spatial impedance dominates accessibility, whereas behavioural capac- ity modulates care effort. The findings illustrate hallmark properties of complex adaptive social systems-emergence, heterogeneity, and feedback-demonstrating how computational social simulation can illuminate policy trade-offs between spatial efficiency, social equity, and care sustainability in ageing territories.

Method: Two-level approach: macro-level pooled ε-machine for system-wide information, micro-level ε-machines per caregiver-elder dyad with compression metrics (normalized LZ78 complexity, bits per symbol). Feature set {hμ, Cμ, E, LZ78, bps} enables distributional analysis and clustering.

Result: Global reconstructions show memoryless baseline, while per-dyad models reveal localized structure, especially for walkability. Compression metrics confirm patterns: dictionary compressors identify redundancy, LZ78 captures statistical novelty. Socioeconomic variables show heterogeneity, spatial interaction induces temporal memory.

Conclusion: The framework successfully distinguishes semantic organization (predictive causation, memory) from syntactic simplicity (description length) and clarifies emergence across system layers, demonstrated through caregiver-elder case study.

Abstract: We propose a two-level information-theoretic framework for characterizing the informational organization of Agent-Based Model (ABM) dynamics within the broader paradigm of Complex Adaptive Systems (CAS). At the macro level, a pooled $\varepsilon$-machine is reconstructed as a reference model summarizing the system-wide informational regime. At the micro level, $\varepsilon$-machines are reconstructed for each caregiver–elder dyad and variable, complemented by algorithm-agnostic Kolmogorov-style measures, including normalized LZ78 complexity and bits per symbol from lossless compression. The resulting feature set, ${h_μ, C_μ, E, \mathrm{LZ78}, \mathrm{bps}}$, enables distributional analysis, stratified comparisons, and unsupervised clustering across agents and scenarios. Empirical results show that coupling $\varepsilon$-machines with compression diagnostics yields a coherent picture of where predictive information resides in the caregiving ABM. Global reconstructions provide a memoryless baseline ($L{=}0$ under coarse symbolizations), whereas per-dyad models reveal localized structure, particularly for walkability under ordinal encodings ($m{=}3$). Compression metrics corroborate these patterns: dictionary compressors agree on algorithmic redundancy, while normalized LZ78 captures statistical novelty. Socioeconomic variables display cross-sectional heterogeneity and near-memoryless dynamics, whereas spatial interaction induces bounded temporal memory and recurrent regimes. The framework thus distinguishes semantic organization (predictive causation and memory) from syntactic simplicity (description length) and clarifies how emergence manifests at different system layers. It is demonstrated on a caregiver–elder case study with dyad-level $\varepsilon$-machine reconstructions and compression-based diagnostics.

Method: Combines Shapley-based credit assignment for success cases with first-error localization for failure cases, producing bounded, cooperative signals compatible with reinforcement or preference-based training.

Result: Produces local, signed, and credit-conserving signals that fairly allocate outcomes, promote cooperation, discourage redundancy/sabotage, and penalize harmful steps while rewarding corrections.

Conclusion: Provides a unified and auditable pathway from global evaluation to local supervision in LLM multi-agent training, with theoretical foundation presented for future empirical validation.

Abstract: Large Language Models (LLMs) in multi-agent systems (MAS) have shown promise for complex tasks, yet current training methods lack principled ways to connect system-level evaluation with agent-level and message-level learning. We propose a theoretical framework that unifies cooperative game-theoretic attribution with process reward modeling to transform system evaluation into agent credit and then into response-level signals. Unlike prior approaches that rely only on attribution (e.g., Shapley) or step-level labels (e.g., PRM), our method produces local, signed, and credit-conserving signals. In success cases, Shapley-based credit assignment fairly allocates outcomes across agents and is refined into per-message rewards that promote cooperation while discouraging redundancy or sabotage. In failure cases, first-error localization yields repair-aware preferences that penalize harmful steps while rewarding corrective attempts. The resulting signals are bounded, cooperative, and directly compatible with reinforcement-based or preference-based post-training, providing a unified and auditable pathway from global evaluation to local supervision in LLM multi-agent training. Our contribution is conceptual: we present a theoretical foundation and training signals, leaving empirical validation for future work.

Method: Used large language models to summarize and draft legal letters based on 120 YouTube videos showing legal issues across various domains.

Result: 71.7% of the generated summaries were rated as high or medium quality, demonstrating promising capability in understanding video content for legal purposes.

Conclusion: LLMs show strong potential for understanding video-based legal events and generating legal documents, opening opportunities for improved access to justice applications.

Abstract: Understanding the legally relevant factual basis of an event and conveying it through text is a key skill of legal professionals. This skill is important for preparing forms (e.g., insurance claims) or other legal documents (e.g., court claims), but often presents a challenge for laypeople. Current AI approaches aim to bridge this gap, but mostly rely on the user to articulate what has happened in text, which may be challenging for many. Here, we investigate the capability of large language models (LLMs) to understand and summarize events occurring in videos. We ask an LLM to summarize and draft legal letters, based on 120 YouTube videos showing legal issues in various domains. Overall, 71.7% of the summaries were rated as of high or medium quality, which is a promising result, opening the door to a number of applications in e.g. access to justice.

Method: TeleEMS comprises client applications (phones, smart glasses, desktops) and an edge server with EMS-Stream for multi-party video streaming. It includes three analytics modules: EMSLlama for audio-to-symptom extraction, rPPG methods for video-to-vital heart rate estimation, and PreNet for joint text-vital analytics predicting EMS protocols and interventions.

Result: EMSLlama outperforms GPT-4o (exact-match 0.89 vs. 0.57) and text-vital fusion improves inference robustness, enabling reliable pre-arrival intervention recommendations.

Conclusion: TeleEMS demonstrates the potential of mobile live video analytics to transform EMS operations by bridging the gap between bystanders, dispatchers, and EMTs, paving the way for next-generation intelligent EMS infrastructure.

Abstract: Timely and accurate pre-arrival video streaming and analytics are critical for emergency medical services (EMS) to deliver life-saving interventions. Yet, current-generation EMS infrastructure remains constrained by one-to-one video streaming and limited analytics capabilities, leaving dispatchers and EMTs to manually interpret overwhelming, often noisy or redundant information in high-stress environments. We present TeleEMS, a mobile live video analytics system that enables pre-arrival multimodal inference by fusing audio and video into a unified decision-making pipeline before EMTs arrive on scene. TeleEMS comprises two key components: TeleEMS Client and TeleEMS Server. The TeleEMS Client runs across phones, smart glasses, and desktops to support bystanders, EMTs en route, and 911 dispatchers. The TeleEMS Server, deployed at the edge, integrates EMS-Stream, a communication backbone that enables smooth multi-party video streaming. On top of EMSStream, the server hosts three real-time analytics modules: (1) audio-to-symptom analytics via EMSLlama, a domain-specialized LLM for robust symptom extraction and normalization; (2) video-to-vital analytics using state-of-the-art rPPG methods for heart rate estimation; and (3) joint text-vital analytics via PreNet, a multimodal multitask model predicting EMS protocols, medication types, medication quantities, and procedures. Evaluation shows that EMSLlama outperforms GPT-4o (exact-match 0.89 vs. 0.57) and that text-vital fusion improves inference robustness, enabling reliable pre-arrival intervention recommendations. TeleEMS demonstrates the potential of mobile live video analytics to transform EMS operations, bridging the gap between bystanders, dispatchers, and EMTs, and paving the way for next-generation intelligent EMS infrastructure.

Method: Uses N specific encoders and one shared encoder to extract subject-specific and subject-invariant information, with a Top-K collaboration module to leverage knowledge from previous subjects.

Result: Demonstrated effective cross-subject decoding and fast new subject adaptation on fMRI/EEG-to-video benchmarks using only one model.

Conclusion: MindCross provides an efficient solution for cross-subject brain decoding that addresses data scarcity while enabling rapid adaptation to new subjects.

Abstract: Reconstructing video from brain signals is an important brain decoding task. Existing brain decoding frameworks are primarily built on a subject-dependent paradigm, which requires large amounts of brain data for each subject. However, the expensive cost of collecting brain-video data causes severe data scarcity. Although some cross-subject methods being introduced, they often overfocus with subject-invariant information while neglecting subject-specific information, resulting in slow fine-tune-based adaptation strategy. To achieve fast and data-efficient new subject adaptation, we propose MindCross, a novel cross-subject framework. MindCross’s N specific encoders and one shared encoder are designed to extract subject-specific and subject-invariant information, respectively. Additionally, a Top-K collaboration module is adopted to enhance new subject decoding with the knowledge learned from previous subjects’ encoders. Extensive experiments on fMRI/EEG-to-video benchmarks demonstrate MindCross’s efficacy and efficiency of cross-subject decoding and new subject adaptation using only one model.

Method: Principled Coarse-Graining (PCG) verifies proposals at Acoustic Similarity Groups level derived from target model’s embedding space, using overlap-aware coarse-grained distribution and rejection sampling.

Result: On LibriTTS, PCG increases acceptance rates and throughput compared to standard speculative decoding and prior speech-specific relaxations while maintaining intelligibility and speaker similarity.

Conclusion: Acoustically aware, group-level acceptance provides a simple and general way to accelerate speech token generation while maintaining quality.

Abstract: Speculative decoding accelerates autoregressive speech generation by letting a fast draft model propose tokens that a larger target model verifies. However, for speech LLMs that generate acoustic tokens, exact token matching is overly restrictive: many discrete tokens are acoustically or semantically interchangeable, reducing acceptance rates and limiting speedups. We introduce Principled Coarse-Graining (PCG), which verifies proposals at the level of Acoustic Similarity Groups (ASGs) derived from the target model’s embedding space. By splitting each token’s probability mass across the overlapping groups that contain it, we define an overlap-aware coarse-grained distribution and perform rejection sampling on the resulting group variable. This yields an exactness guarantee at the group level while allowing the accepted draft token to stand in for any member of the group in practice. On LibriTTS, PCG increases acceptance and throughput relative to standard speculative decoding and prior speech-specific relaxations while maintaining intelligibility and speaker similarity. These results suggest acoustically aware, group-level acceptance as a simple and general way to accelerate speech token generation while maintaining speech quality.

Method: Uses Bayesian Optimization with CLAP-based score function to search for optimal audio effect configurations, and introduces a guiding prompt to prevent artifacts and enhance human preference.

Result: The method achieves highest scores that align closely with human preferences, as evaluated through a proposed AI-based evaluation framework.

Conclusion: FxSearcher effectively discovers optimal audio effect configurations for text-guided audio transformations, overcoming limitations of previous differentiable methods.

Abstract: Achieving diverse and high-quality audio transformations from text prompts remains challenging, as existing methods are fundamentally constrained by their reliance on a limited set of differentiable audio effects. This paper proposes \textbf{FxSearcher}, a novel gradient-free framework that discovers the optimal configuration of audio effects (FX) to transform a source signal according to a text prompt. Our method employs Bayesian Optimization and CLAP-based score function to perform this search efficiently. Furthermore, a guiding prompt is introduced to prevent undesirable artifacts and enhance human preference. To objectively evaluate our method, we propose an AI-based evaluation framework. The results demonstrate that the highest scores achieved by our method on these metrics align closely with human preferences. Demos are available at https://hojoonki.github.io/FxSearcher/

Method: Developed TTA model trained on 358k hours of multilingual speech data across ASR, speech translation, and speech-text alignment tasks to produce robust cross-lingual speech representations.

Result: TTA demonstrates superiority over Whisper across ASR/ST, speech retrieval, and ASR-LLM performance benchmarks, with validated cross-lingual capabilities.

Conclusion: TTA provides more effective speech representations for LLM integration and will be released as part of the Auden audio understanding toolkit.

Abstract: Speech-LLM models have demonstrated great performance in multi-modal and multi-task speech understanding. A typical speech-LLM paradigm is integrating speech modality with a large language model (LLM). While the Whisper encoder was frequently adopted in previous studies for speech input, it shows limitations regarding input format, model scale, and semantic performance. To this end, we propose a lightweight TTA model specialized in speech semantics for more effective LLM integration. With large-scale training of 358k hours of speech data on multilingual speech recognition (ASR), speech translation (ST) and speech-text alignment tasks, TTA is capable of producing robust cross-lingual speech representations. Extensive evaluations across diverse benchmarks, including ASR/ST, speech retrieval, and ASR-LLM performance assessments, demonstrate TTA’s superiority over Whisper. Furthermore, we rigorously validate the interplay between cross-lingual capabilities and ASR/ST performance. The model weights and training recipes of TTA will be released as part of an audio understanding toolkit Auden.

Method: NDF computes a single-channel complex mask from microphone array signals and applies it to a reference microphone to approximate a virtual directional microphone with desired directivity pattern.

Result: NDF achieves frequency-invariant directivity patterns above spatial aliasing frequency, approximates diverse higher-order patterns, enables pattern steering, and generalizes to unseen conditions.

Conclusion: The proposed neural directional filtering approach demonstrates superior performance over conventional beamforming and parametric methods for achieving desired directivity patterns with compact arrays.

Abstract: Beamforming with desired directivity patterns using compact microphone arrays is essential in many audio applications. Directivity patterns achievable using traditional beamformers depend on the number of microphones and the array aperture. Generally, their effectiveness degrades for compact arrays. To overcome these limitations, we propose a neural directional filtering (NDF) approach that leverages deep neural networks to enable sound capture with a predefined directivity pattern. The NDF computes a single-channel complex mask from the microphone array signals, which is then applied to a reference microphone to produce an output that approximates a virtual directional microphone with the desired directivity pattern. We introduce training strategies and propose data-dependent metrics to evaluate the directivity pattern and directivity factor. We show that the proposed method: i) achieves a frequency-invariant directivity pattern even above the spatial aliasing frequency, ii) can approximate diverse and higher-order patterns, iii) can steer the pattern in different directions, and iv) generalizes to unseen conditions. Lastly, experimental comparisons demonstrate superior performance over conventional beamforming and parametric approaches.

Method: Evaluated a convolutional neural network (CNN) model with various combinations of amplitude features (magnitude spectrogram, ILD) and phase features (phase spectrogram, IPD) on both in-domain and out-of-domain data with mismatched HRTFs.

Result: Carefully chosen feature combinations often outperform model complexity increases; ILD+IPD works for in-domain SSL, while generalization requires channel spectrograms with ILD and IPD; low-complexity CNN achieves competitive performance with optimal features.

Conclusion: Feature design is crucial for binaural SSL, with practical guidance provided for both domain-specific and general-purpose localization applications.

Abstract: This study presents a systematic evaluation of time-frequency feature design for binaural sound source localization (SSL), focusing on how feature selection influences model performance across diverse conditions. We investigate the performance of a convolutional neural network (CNN) model using various combinations of amplitude-based features (magnitude spectrogram, interaural level difference - ILD) and phase-based features (phase spectrogram, interaural phase difference - IPD). Evaluations on in-domain and out-of-domain data with mismatched head-related transfer functions (HRTFs) reveal that carefully chosen feature combinations often outperform increases in model complexity. While two-feature sets such as ILD + IPD are sufficient for in-domain SSL, generalization to diverse content requires richer inputs combining channel spectrograms with both ILD and IPD. Using the optimal feature sets, our low-complexity CNN model achieves competitive performance. Our findings underscore the importance of feature design in binaural SSL and provide practical guidance for both domain-specific and general-purpose localization.

Method: SCOPE combines Adaptive Codebook Compression (ACC) for frequency-encoded latent representations tailored to sonar, with Frequency-Aware Multiscale Segmentation (FAMS) that decomposes frames into low-frequency structure and sparse high-frequency dynamics while suppressing artifacts. Uses hedging training strategy with low-pass proxy pairs.

Result: Achieves SSIM of 0.77 (40% improvement over prior self-supervised baselines) at bitrates ≤0.0118 bpp, reduces uplink bandwidth by >80%, improves downstream detection, runs in real-time (3.1 ms encoding on embedded GPU, 97 ms decoding on server). Successfully deployed for months in three Pacific Northwest rivers.

Conclusion: Learning frequency-structured latents enables practical, low-bitrate sonar streaming with preserved signal details under real-world deployment conditions, supporting real-time salmon enumeration and environmental monitoring.

Abstract: Real-time imaging sonar has become an important tool for underwater monitoring in environments where optical sensing is unreliable. Its broader use is constrained by two coupled challenges: highly limited uplink bandwidth and severe sonar-specific artifacts (speckle, motion blur, reverberation, acoustic shadows) that affect up to 98% of frames. We present SCOPE, a self-supervised framework that jointly performs compression and artifact correction without clean-noise pairs or synthetic assumptions. SCOPE combines (i) Adaptive Codebook Compression (ACC), which learns frequency-encoded latent representations tailored to sonar, with (ii) Frequency-Aware Multiscale Segmentation (FAMS), which decomposes frames into low-frequency structure and sparse high-frequency dynamics while suppressing rapidly fluctuating artifacts. A hedging training strategy further guides frequency-aware learning using low-pass proxy pairs generated without labels. Evaluated on months of in-situ ARIS sonar data, SCOPE achieves a structural similarity index (SSIM) of 0.77, representing a 40% improvement over prior self-supervised denoising baselines, at bitrates down to <= 0.0118 bpp. It reduces uplink bandwidth by more than 80% while improving downstream detection. The system runs in real time, with 3.1 ms encoding on an embedded GPU and 97 ms full multi-layer decoding on the server end. SCOPE has been deployed for months in three Pacific Northwest rivers to support real-time salmon enumeration and environmental monitoring in the wild. Results demonstrate that learning frequency-structured latents enables practical, low-bitrate sonar streaming with preserved signal details under real-world deployment conditions.

Method: Reformulates PFGM++ into a conditional generative framework by incorporating sparse-view data as guidance during training and sampling, modeling the posterior distribution of full-view reconstructions conditioned on sparse observations.

Result: Outperforms baselines with improved artifact suppression, enhanced detail preservation, and reliable performance in dose-sensitive and time-critical imaging scenarios.

Conclusion: PoCGM successfully adapts PFGM++ for medical imaging tasks, providing an effective solution for sparse-view CT reconstruction that balances radiation reduction with image quality preservation.

Abstract: In computed tomography (CT), reducing the number of projection views is an effective strategy to lower radiation exposure and/or improve temporal resolution. However, this often results in severe aliasing artifacts and loss of structural details in reconstructed images, posing significant challenges for clinical applications. Inspired by the success of the Poisson Flow Generative Model (PFGM++) in natural image generation, we propose a PoCGM (Poisson-Conditioned Generative Model) to address the challenges of sparse-view CT reconstruction. Since PFGM++ was originally designed for unconditional generation, it lacks direct applicability to medical imaging tasks that require integrating conditional inputs. To overcome this limitation, the PoCGM reformulates PFGM++ into a conditional generative framework by incorporating sparse-view data as guidance during both training and sampling phases. By modeling the posterior distribution of full-view reconstructions conditioned on sparse observations, PoCGM effectively suppresses artifacts while preserving fine structural details. Qualitative and quantitative evaluations demonstrate that PoCGM outperforms the baselines, achieving improved artifact suppression, enhanced detail preservation, and reliable performance in dose-sensitive and time-critical imaging scenarios.

Method: Combines cross-bit-depth feature propagation (reusing features from denser depths), Bag-of-Encoders selection (choosing optimal network per depth from small pool), and Morton-order-preserving hierarchy (maintaining global Z-order across transitions).

Result: Achieves state-of-the-art compression at real-time throughput on Ford and SemanticKITTI datasets.

Conclusion: The proposed framework improves entropy modeling and computational efficiency for hierarchical LiDAR geometry compression.

Abstract: Hierarchical LiDAR geometry compression encodes voxel occupancies from low to high bit-depths, yet prior methods treat each depth independently and re-estimate local context from coordinates at every level, limiting compression efficiency. We present ELiC, a real-time framework that combines cross-bit-depth feature propagation, a Bag-of-Encoders (BoE) selection scheme, and a Morton-order-preserving hierarchy. Cross-bit-depth propagation reuses features extracted at denser, lower depths to support prediction at sparser, higher depths. BoE selects, per depth, the most suitable coding network from a small pool, adapting capacity to observed occupancy statistics without training a separate model for each level. The Morton hierarchy maintains global Z-order across depth transitions, eliminating per-level sorting and reducing latency. Together these components improve entropy modeling and computation efficiency, yielding state-of-the-art compression at real-time throughput on Ford and SemanticKITTI. Code and models will be released upon publication.

Method: Proposes NEtwork-Regularized diffusion sampling (NERD) with L1 regularization for spatial continuity across CT slices, using ADMM and PDHG optimization strategies to solve the objective.

Result: NERD achieves state-of-the-art or highly competitive results on medical 3D CT data, effectively reducing inter-slice artifacts and producing coherent volumetric reconstructions.

Conclusion: The proposed NERD framework successfully extends diffusion-based reconstruction to 3D CT by incorporating spatial regularization, demonstrating superior performance for volumetric medical imaging tasks.

Abstract: Numerous diffusion model (DM)-based methods have been proposed for solving inverse imaging problems. Among these, a recent line of work has demonstrated strong performance by formulating sampling as an optimization procedure that enforces measurement consistency, forward diffusion consistency, and both step-wise and backward diffusion consistency. However, these methods have only considered 2D reconstruction tasks and do not directly extend to 3D image reconstruction problems, such as in Computed Tomography (CT). To bridge this gap, we propose NEtwork-Regularized diffusion sampling for 3D CT (NERD) by incorporating an L1 regularization into the optimization objective. This regularizer encourages spatial continuity across adjacent slices, reducing inter-slice artifacts and promoting coherent volumetric reconstructions. Additionally, we introduce two efficient optimization strategies to solve the resulting objective: one based on the Alternating Direction Method of Multipliers (ADMM) and another based on the Primal-Dual Hybrid Gradient (PDHG) method. Experiments on medical 3D CT data demonstrate that our approach achieves either state-of-the-art or highly competitive results.

Method: Created a new IQA dataset for SR broadcast images in 2K and 4K resolutions, conducted subjective quality evaluation to obtain Mean Opinion Scores (MOS), performed comprehensive human study to identify key quality factors, and evaluated existing IQA metrics on the dataset.

Result: The study revealed limitations of current IQA metrics, showing they don’t adequately correlate with perceived quality of SR images, particularly when assessing SR-enhanced content without original high-quality references.

Conclusion: There is a need for more robust IQA metrics that better correlate with human perception of SR image quality, especially for broadcast content enhancement scenarios.

Abstract: To display low-quality broadcast content on high-resolution screens in full-screen format, the application of Super-Resolution (SR), a key consumer technology, is essential. Recently, SR methods have been developed that not only increase resolution while preserving the original image information but also enhance the perceived quality. However, evaluating the quality of SR images generated from low-quality sources, such as SR-enhanced broadcast content, is challenging due to the need to consider both distortions and improvements. Additionally, assessing SR image quality without original high-quality sources presents another significant challenge. Unfortunately, there has been a dearth of research specifically addressing the Image Quality Assessment (IQA) of SR images under these conditions. In this work, we introduce a new IQA dataset for SR broadcast images in both 2K and 4K resolutions. We conducted a subjective quality evaluation to obtain the Mean Opinion Score (MOS) for these SR images and performed a comprehensive human study to identify the key factors influencing the perceived quality. Finally, we evaluated the performance of existing IQA metrics on our dataset. This study reveals the limitations of current metrics, highlighting the need for a more robust IQA metric that better correlates with the perceived quality of SR images.

Method: Weakly supervised learning algorithm that iteratively generates soft segmentation maps through model explainability, masking detected regions and repeating to discover additional embolisms.

Result: Achieved excellent PE detection performance with significant improvements at each iteration, comparable to strongly supervised methods on RSPECT dataset.

Conclusion: iExplain effectively transforms coarse annotations into detailed PE masks through iterative refinement, outperforming existing weakly supervised methods.

Abstract: Pulmonary Embolism (PE) are a leading cause of cardiovascular death. Computed tomographic pulmonary angiography (CTPA) is the gold standard for PE diagnosis, with growing interest in AI-based diagnostic assistance. However, these algorithms are limited by scarce fine-grained annotations of thromboembolic burden. We address this challenge with iExplain, a weakly supervised learning algorithm that transforms coarse image-level annotations into detailed pixel-level PE masks through iterative model explainability. Our approach generates soft segmentation maps used to mask detected regions, enabling the process to repeat and discover additional embolisms that would be missed in a single pass. This iterative refinement effectively captures complete PE regions and detects multiple distinct embolisms. Models trained on these automatically generated annotations achieve excellent PE detection performance, with significant improvements at each iteration. We demonstrate iExplain’s effectiveness on the RSPECT augmented dataset, achieving results comparable to strongly supervised methods while outperforming existing weakly supervised methods.

Method: Introduces imaging Knowledge-Informed Dynamic Optimal Transport (KIDOT) that models reconstruction as finding a dynamic transport path from measurements to images, guided by an imaging knowledge-informed cost function and transport equation, learning from unpaired data.

Result: Extensive experiments on MRI and CT reconstruction demonstrate KIDOT’s superior performance compared to existing methods.

Conclusion: KIDOT provides a mathematically rigorous framework that enhances robustness in medical image reconstruction by better leveraging unpaired data while respecting acquisition physics, effectively bridging the retrospective-to-prospective gap.

Abstract: Medical image reconstruction from measurement data is a vital but challenging inverse problem. Deep learning approaches have achieved promising results, but often requires paired measurement and high-quality images, which is typically simulated through a forward model, i.e., retrospective reconstruction. However, training on simulated pairs commonly leads to performance degradation on real prospective data due to the retrospective-to-prospective gap caused by incomplete imaging knowledge in simulation. To address this challenge, this paper introduces imaging Knowledge-Informed Dynamic Optimal Transport (KIDOT), a novel dynamic optimal transport framework with optimality in the sense of preserving consistency with imaging physics in transport, that conceptualizes reconstruction as finding a dynamic transport path. KIDOT learns from unpaired data by modeling reconstruction as a continuous evolution path from measurements to images, guided by an imaging knowledge-informed cost function and transport equation. This dynamic and knowledge-aware approach enhances robustness and better leverages unpaired data while respecting acquisition physics. Theoretically, we demonstrate that KIDOT naturally generalizes dynamic optimal transport, ensuring its mathematical rationale and solution existence. Extensive experiments on MRI and CT reconstruction demonstrate KIDOT’s superior performance.

Method: Review of over 150 studies examining FM development and application in medical imaging, covering model architectures, self-supervised learning methods, and downstream adaptation strategies across pathology, radiology, and ophthalmology domains.

Result: Comprehensive examination of how FMs are being implemented across different medical imaging specialties, with comparisons of design choices and application approaches.

Conclusion: Foundation models show promise for medical image analysis but face key challenges and open questions that need to be addressed in future research to fully realize their potential.

Abstract: Foundation models (FMs) are changing the way medical images are analyzed by learning from large collections of unlabeled data. Instead of relying on manually annotated examples, FMs are pre-trained to learn general-purpose visual features that can later be adapted to specific clinical tasks with little additional supervision. In this review, we examine how FMs are being developed and applied in pathology, radiology, and ophthalmology, drawing on evidence from over 150 studies. We explain the core components of FM pipelines, including model architectures, self-supervised learning methods, and strategies for downstream adaptation. We also review how FMs are being used in each imaging domain and compare design choices across applications. Finally, we discuss key challenges and open questions to guide future research.

Method: Proposes DeSamba framework with Decoupled Representation Learning Module (DRLM) for feature decoupling through self-reconstruction and cross-reconstruction, and Spectral Adaptive Modulation Block (SAMB) within SAMNet for dynamic fusion of spectral and spatial information based on lesion characteristics.

Result: On spinal metastasis dataset (n=1,448): 62.10% Top-1 accuracy, 63.62% F1-score, 87.71% AUC, 93.55% Top-3 accuracy on external validation (n=372). On spondylitis dataset (n=251): 70.00%/64.52% accuracy and 74.75/73.88 AUC on internal/external validation. Outperforms all SOTA baselines with over 10% relative improvement.

Conclusion: DeSamba demonstrates potential as a generalizable and effective solution for 3D lesion classification in multi-sequence medical imaging, with both DRLM and SAMB significantly contributing to performance.

Abstract: Magnetic Resonance Imaging (MRI) sequences provide rich spatial and frequency domain information, which is crucial for accurate lesion classification in medical imaging. However, effectively integrating multi-sequence MRI data for robust 3D lesion classification remains a challenge. In this paper, we propose DeSamba (Decoupled Spectral Adaptive Network and Mamba-Based Model), a novel framework designed to extract decoupled representations and adaptively fuse spatial and spectral features for lesion classification. DeSamba introduces a Decoupled Representation Learning Module (DRLM) that decouples features from different MRI sequences through self-reconstruction and cross-reconstruction, and a Spectral Adaptive Modulation Block (SAMB) within the proposed SAMNet, enabling dynamic fusion of spectral and spatial information based on lesion characteristics. We evaluate DeSamba on two clinically relevant 3D datasets. On a six-class spinal metastasis dataset (n=1,448), DeSamba achieves 62.10% Top-1 accuracy, 63.62% F1-score, 87.71% AUC, and 93.55% Top-3 accuracy on an external validation set (n=372), outperforming all state-of-the-art (SOTA) baselines. On a spondylitis dataset (n=251) involving a challenging binary classification task, DeSamba achieves 70.00%/64.52% accuracy and 74.75/73.88 AUC on internal and external validation sets, respectively. Ablation studies demonstrate that both DRLM and SAMB significantly contribute to overall performance, with over 10% relative improvement compared to the baseline. Our results highlight the potential of DeSamba as a generalizable and effective solution for 3D lesion classification in multi-sequence medical imaging.

Method: Uses hardware-software co-design with sparse priors in predefined bases and generative priors from denoising diffusion models to reconstruct images from undersampled spatial and Fourier-domain measurements.

Result: Achieves up to 2.5x acquisition throughput increase while preserving original 3D resolution, with controllable trade-offs between measurement count and downsampling level.

Conclusion: cryoSENSE enables compressive cryo-EM sensing with significant throughput improvements, where sparse priors work best for Fourier-domain measurements and generative priors excel with pixel-domain measurements and severe undersampling.

Abstract: Cryo-electron microscopy (cryo-EM) enables the atomic-resolution visualization of biomolecules; however, modern direct detectors generate data volumes that far exceed the available storage and transfer bandwidth, thereby constraining practical throughput. We introduce cryoSENSE, the computational realization of a hardware-software co-designed framework for compressive cryo-EM sensing and acquisition. We show that cryo-EM images of proteins lie on low-dimensional manifolds that can be independently represented using sparse priors in predefined bases and generative priors captured by a denoising diffusion model. cryoSENSE leverages these low-dimensional manifolds to enable faithful image reconstruction from spatial and Fourier-domain undersampled measurements while preserving downstream structural resolution. In experiments, cryoSENSE increases acquisition throughput by up to 2.5$\times$ while retaining the original 3D resolution, offering controllable trade-offs between the number of masked measurements and the level of downsampling. Sparse priors favor faithful reconstruction from Fourier-domain measurements and moderate compression, whereas generative diffusion priors achieve accurate recovery from pixel-domain measurements and more severe undersampling. Project website: https://cryosense.github.io.