Method: Developed Collaborative Battleship task to study exploration-exploitation tradeoffs, then created Monte Carlo inference strategies based on Bayesian Experimental Design principles for both Spotter (answer generation) and Captain (question selection) agents.

Result: BED methods boosted Spotter accuracy by 14.7%, increased Captain information gain by 0.227 bits (94.2% of noise ceiling), improved targeting by 0.303-0.374 F1, and enabled weaker LMs to outperform humans (82% win rate) and frontier models (67% win rate vs GPT-5) at 1% cost.

Conclusion: The Bayesian Experimental Design approach significantly enhances LM agents’ rational information-seeking capabilities, demonstrating general applicability across different strategic dialogue tasks like Guess Who? with substantial accuracy improvements.

Abstract: Many high-stakes applications of AI require forming data-driven hypotheses and making targeted guesses; e.g., in scientific and diagnostic settings. Given limited resources, to what extent do agents based on language models (LMs) act rationally? We develop methods to benchmark and enhance agentic information-seeking, drawing on insights from human behavior. First, we introduce a strategic decision-oriented dialogue task called Collaborative Battleship, in which a partially-informed Captain must balance exploration (asking questions) and action (taking shots), while a fully-informed Spotter must provide accurate answers under an information bottleneck. Compared to human players (N=42), we find that LM agents struggle to ground answers in context, generate informative questions, and select high-value actions. Next, to address these gaps, we develop novel Monte Carlo inference strategies for LMs based on principles from Bayesian Experimental Design (BED). For Spotter agents, our approach boosts accuracy by up to 14.7% absolute over LM-only baselines; for Captain agents, it raises expected information gain (EIG) by up to 0.227 bits (94.2% of the achievable noise ceiling). Combined, these components yield sharper targeting (+0.303-0.374 F1), and enable weaker LMs, such as Llama-4-Scout, to outperform both humans (8% -> 82% win rate) and frontier models (0% -> 67% win rate vs. GPT-5) at ~1% of GPT-5’s cost. We replicate these findings on Guess Who? where our methods significantly boost accuracy (+28.3-42.4 p.p.), demonstrating their general applicability for building rational information-seeking agents.

Method: CodeAdapt combines CodeAct framework (interleaving natural language reasoning with code execution) with few-shot bootstrap in-context learning from just 5 training problems.

Result: CodeAdapt enabled 3 LMs to outperform corresponding reasoning models on average over 8 tasks (up to 22.9%) while being 10-81% more token efficient, and delivered superior performance on 6 tasks when averaged over 4 models (up to 35.7%).

Conclusion: CodeAdapt-style learning is robust and domain general, and code-enabled LMs are cognitively grounded systems that provide a strong foundation for in-weight reinforcement learning.

Abstract: Reasoning models (RMs), language models (LMs) trained with reinforcement learning to produce long-form natural language reasoning, have been remarkably successful, but they still require large amounts of computation and data to train, and can be slow and expensive to run. In this paper, we show that standard instruct LMs can already be elicited to be strong reasoners at a level comparable to or even surpassing their corresponding RMs (e.g., DeepSeek V3 vs R1) without finetuning, across diverse domains from instruction following and creative generation to mathematical reasoning. This is achieved by CodeAdapt, our simple recipe that combines the CodeAct framework, where LMs interleave natural language reasoning with code execution in a multi-step fashion, with few-shot bootstrap in-context learning from as few as five training problems. Analyzing four matched pairs of LMs and RMs, we find that CodeAdapt enables three LMs to outperform the corresponding RMs on average over eight tasks (up to 22.9%) while being 10-81% more token efficient, and delivers superior performance on six tasks when averaged over the four models (up to 35.7%). Furthermore, the code-augmented reasoning traces display rich and varied problem-solving strategies. Our findings support that (1) CodeAdapt-style learning and reasoning may be robust and domain general and (2) code-enabled LMs are cognitively grounded and powerful systems, potentially providing a strong foundation for in-weight reinforcement learning.

Method: Assembled dataset of long-form, recently written fiction with author-produced metadata, validated by digital humanities scholars, and evaluated embedding models on 12 similarity axes.

Result: Embedding models tend to focus on surface-level features rather than semantic categories useful for computational literary studies.

Conclusion: FICSIM provides a suitable benchmark for literary-domain tasks while prioritizing author agency and informed consent throughout data collection.

Abstract: As language models become capable of processing increasingly long and complex texts, there has been growing interest in their application within computational literary studies. However, evaluating the usefulness of these models for such tasks remains challenging due to the cost of fine-grained annotation for long-form texts and the data contamination concerns inherent in using public-domain literature. Current embedding similarity datasets are not suitable for evaluating literary-domain tasks because of a focus on coarse-grained similarity and primarily on very short text. We assemble and release FICSIM, a dataset of long-form, recently written fiction, including scores along 12 axes of similarity informed by author-produced metadata and validated by digital humanities scholars. We evaluate a suite of embedding models on this task, demonstrating a tendency across models to focus on surface-level features over semantic categories that would be useful for computational literary studies tasks. Throughout our data-collection process, we prioritize author agency and rely on continual, informed author consent.

Method: Assessment of state-of-the-art LLMs on identifying overruling relationships from 236 U.S. Supreme Court case pairs, focusing on long-document legal understanding.

Result: Three critical limitations found: era sensitivity (degraded performance on historical cases), shallow reasoning (reliance on heuristics rather than deep comprehension), and context-dependent reasoning failures (temporally impossible relationships in complex tasks).

Conclusion: The work contributes a benchmark that addresses the critical gap in realistic long-context evaluation, providing an environment that mirrors the complexity and stakes of actual legal reasoning tasks.

Abstract: Large language models (LLMs) with extended context windows show promise for complex legal reasoning tasks, yet their ability to understand long legal documents remains insufficiently evaluated. Developing long-context benchmarks that capture realistic, high-stakes tasks remains a significant challenge in the field, as most existing evaluations rely on simplified synthetic tasks that fail to represent the complexity of real-world document understanding. Overruling relationships are foundational to common-law doctrine and commonly found in judicial opinions. They provide a focused and important testbed for long-document legal understanding that closely resembles what legal professionals actually do. We present an assessment of state-of-the-art LLMs on identifying overruling relationships from U.S. Supreme Court cases using a dataset of 236 case pairs. Our evaluation reveals three critical limitations: (1) era sensitivity – the models show degraded performance on historical cases compared to modern ones, revealing fundamental temporal bias in their training; (2) shallow reasoning – models rely on shallow logical heuristics rather than deep legal comprehension; and (3) context-dependent reasoning failures – models produce temporally impossible relationships in complex open-ended tasks despite maintaining basic temporal awareness in simple contexts. Our work contributes a benchmark that addresses the critical gap in realistic long-context evaluation, providing an environment that mirrors the complexity and stakes of actual legal reasoning tasks.

Method: Manually constructed 1020 minimal pairs across 11 linguistic features through a team of fluent Irish speakers, then evaluated both existing LLMs and human participants on their syntactic knowledge.

Result: Humans outperformed all models by 16.6% average accuracy (90.1% vs 73.5% for best model). A 18.1% gap exists between open- and closed-source LLMs. Humans and models struggled on different grammatical aspects, indicating different learned representations.

Conclusion: Irish-BLiMP provides the first systematic framework for evaluating LLM grammatical competence in Irish and offers a valuable benchmark for advancing research on linguistic understanding in low-resource languages.

Abstract: We present Irish-BLiMP (Irish Benchmark of Linguistic Minimal Pairs), the first dataset and framework designed for fine-grained evaluation of linguistic competence in the Irish language, an endangered language. Drawing on a variety of linguistic literature and grammar reference works, we manually constructed and reviewed 1020 minimal pairs across a taxonomy of 11 linguistic features, through a team of fluent Irish speakers. We evaluate both existing Large Language Models (LLMs) and fluent human participants on their syntactic knowledge of Irish. Our findings show that humans outperform all models across all linguistic features, achieving 16.6% higher accuracy on average. Moreover, a substantial performance gap of 18.1% persists between open- and closed-source LLMs, with even the strongest model (gpt-5) reaching only 73.5% accuracy compared to 90.1% by human. Interestingly, human participants and models struggle on different aspects of Irish grammar, thus highlighting a difference in representation learned by the models. Overall, Irish-BLiMP provides the first systematic framework for evaluating the grammatical competence of LLMs in Irish and offers a valuable benchmark for advancing research on linguistic understanding in low-resource languages.

Method: Systematic study of four training-free confidence estimation methods for CoT gating, comparing them to random baseline and oracle. Evaluates when models should invoke reasoning based on confidence in direct answers.

Result: Existing training-free confidence measures can reduce redundant CoT and outperform randomly invoked CoT. However, utility varies with dataset and model, making practical deployment challenging.

Conclusion: Current confidence-gated CoT methods show potential but have limitations due to inconsistent performance across datasets and models, highlighting the need for more reliable adaptive gating approaches.

Abstract: Chain-of-thought (CoT) prompting has emerged as a common technique for enhancing the reasoning abilities of large language models (LLMs). While extended reasoning can boost accuracy on complex tasks, it is often unnecessary and substantially increases token usage, limiting the practicality of reasoning models in many scenarios. Recent models, such as GPT-OSS and Qwen3, expose controls that enable users to adjust the length of CoT or determine whether it is used at all. Yet, it remains unclear when CoT should be used: on some tasks it improves performance, while on others it provides little benefit or even harms performance. We address this challenge with confidence-gated CoT, where a model invokes reasoning only when confidence in its direct answer is low. To this end, we present the first systematic study of training-free confidence estimation methods for CoT gating. Specifically, we evaluate four training-free confidence estimation methods and compare them to a random baseline and an oracle that always knows when CoT is needed. Through extensive experiments, we show that existing training-free confidence measures can reduce redundant CoT and outperform randomly invoked CoT. However, the utility of individual confidence measures is inconsistent, varying with both the dataset and the model, underscoring the difficulty of deploying confidence-gated CoT in practice. By analysing both strengths and failure modes, our study highlights the potential and limitations of current methods and paves the way toward more reliable adaptive gating of CoT.

Method: Manual annotation of 3,312 factual errors across 180 game summaries generated by Llama-3.1-70B and Qwen2.5-72B models using three input formats: row-structured, JSON, and unstructured NBA play-by-play data.

Result: JSON input reduced error rates by 69% for Llama and 65% for Qwen compared to unstructured input; row-structured input reduced errors by 54% for Llama and 51% for Qwen. Input structure accounted for over 80% of variance in error rates.

Conclusion: Input structure significantly impacts factual accuracy in LLM-generated sports summaries, with structured formats (especially JSON) dramatically reducing hallucinations and factual errors.

Abstract: A major concern when deploying LLMs in accuracy-critical domains such as sports reporting is that the generated text may not faithfully reflect the input data. We quantify how input structure affects hallucinations and other factual errors in LLM-generated summaries of NBA play-by-play data, across three formats: row-structured, JSON and unstructured. We manually annotated 3,312 factual errors across 180 game summaries produced by two models, Llama-3.1-70B and Qwen2.5-72B. Input structure has a strong effect: JSON input reduces error rates by 69% for Llama and 65% for Qwen compared to unstructured input, while row-structured input reduces errors by 54% for Llama and 51% for Qwen. A two-way repeated measures ANOVA shows that input structure accounts for over 80% of the variance in error rates, with Tukey HSD post hoc tests confirming statistically significant differences between all input formats.

Method: Analyzed safety detection and hallucination detection tasks in fine-tuned and zero-shot settings using standard LLMs and Large Reasoning Models, comparing reasoning-augmented (Think On) vs non-reasoning (Think Off) approaches.

Result: Think On improves overall accuracy but underperforms at low-FPR thresholds; Think Off dominates in precision-sensitive regimes; token-based scoring outperforms self-verbalized confidence; ensemble approach recovers strengths of both modes.

Conclusion: Reasoning is a double-edged tool - beneficial for average accuracy but often ill-suited for applications requiring strict precision.

Abstract: Reasoning has become a central paradigm for large language models (LLMs), consistently boosting accuracy across diverse benchmarks. Yet its suitability for precision-sensitive tasks remains unclear. We present the first systematic study of reasoning for classification tasks under strict low false positive rate (FPR) regimes. Our analysis covers two tasks–safety detection and hallucination detection–evaluated in both fine-tuned and zero-shot settings, using standard LLMs and Large Reasoning Models (LRMs). Our results reveal a clear trade-off: Think On (reasoning-augmented) generation improves overall accuracy, but underperforms at the low-FPR thresholds essential for practical use. In contrast, Think Off (no reasoning during inference) dominates in these precision-sensitive regimes, with Think On surpassing only when higher FPRs are acceptable. In addition, we find token-based scoring substantially outperforms self-verbalized confidence for precision-sensitive deployments. Finally, a simple ensemble of the two modes recovers the strengths of each. Taken together, our findings position reasoning as a double-edged tool: beneficial for average accuracy, but often ill-suited for applications requiring strict precision.

Method: Trains a BERT retriever with REINFORCE to rank demonstrations by editing reward, and uses a learnable threshold to prune low-value examples, dynamically adjusting prompt length based on task difficulty.

Result: On COUNTERFACT benchmark: improves edit success by up to 17.1%, reduces latency by 41.6%, and preserves accuracy on unrelated queries.

Conclusion: DR-IKE enables scalable and adaptive knowledge editing without modifying model weights, making it compatible with black-box LLMs.

Abstract: Large language models (LLMs) excel at factual recall yet still propagate stale or incorrect knowledge. In-context knowledge editing offers a gradient-free remedy suitable for black-box APIs, but current editors rely on static demonstration sets chosen by surface-level similarity, leading to two persistent obstacles: (i) a quantity-quality trade-off, and (ii) lack of adaptivity to task difficulty. We address these issues by dynamically selecting supporting demonstrations according to their utility for the edit. We propose Dynamic Retriever for In-Context Knowledge Editing (DR-IKE), a lightweight framework that (1) trains a BERT retriever with REINFORCE to rank demonstrations by editing reward, and (2) employs a learnable threshold to prune low-value examples, shortening the prompt when the edit is easy and expanding it when the task is hard. DR-IKE performs editing without modifying model weights, relying solely on forward passes for compatibility with black-box LLMs. On the COUNTERFACT benchmark, it improves edit success by up to 17.1%, reduces latency by 41.6%, and preserves accuracy on unrelated queries, demonstrating scalable and adaptive knowledge editing. The code is available at https://github.com/mwnafee/DR-IKE .

Method: Uses Adaptive RAG system with a classifier to distinguish question complexity and determine answering strategy, employing machine translation for data augmentation due to limited Indonesian datasets.

Result: Reliable question complexity classifier was developed, but significant inconsistencies in multi-retrieval answering strategy negatively impacted overall evaluation.

Conclusion: The study shows both promise and challenges for QA in low-resource languages, highlighting directions for future improvement despite the multi-retrieval strategy issues.

Abstract: Question Answering (QA) has seen significant improvements with the advancement of machine learning models, further studies enhanced this question answering system by retrieving external information, called Retrieval-Augmented Generation (RAG) to produce more accurate and informative answers. However, these state-of-the-art-performance is predominantly in English language. To address this gap we made an effort of bridging language gaps by incorporating Adaptive RAG system to Indonesian language. Adaptive RAG system integrates a classifier whose task is to distinguish the question complexity, which in turn determines the strategy for answering the question. To overcome the limited availability of Indonesian language dataset, our study employs machine translation as data augmentation approach. Experiments show reliable question complexity classifier; however, we observed significant inconsistencies in multi-retrieval answering strategy which negatively impacted the overall evaluation when this strategy was applied. These findings highlight both the promise and challenges of question answering in low-resource language suggesting directions for future improvement.

Method: Created CDrugRed dataset with comprehensive patient information and benchmarked state-of-the-art LLMs on discharge medication recommendation task using supervised fine-tuning.

Result: Best model achieved F1 score of 0.5648 and Jaccard score of 0.4477, showing substantial room for improvement despite supervised fine-tuning.

Conclusion: CDrugRed establishes a challenging benchmark for clinical drug recommendation and serves as a valuable resource for developing more robust and accurate systems.

Abstract: Intelligent drug recommendation based on Electronic Health Records (EHRs) is critical for improving for improving the quality and efficiency of clinical decision-making. By leveraging large-scale patient data, drug recommendation systems can assist physicians in selecting the most appropriate medications according to a patient’s medical history, diagnoses, laboratory results, and comorbidities. However, the advancement of such systems is significantly hampered by the scarcity of publicly available, real-world EHR datasets, particularly in languages other than English. In this work, we present CDrugRed, a first publicly available Chinese drug recommendation dataset focused on discharge medications for metabolic diseases. The dataset includes 5,894 de-identified records from 3,190 patients, containing comprehensive information such as patient demographics, medical history, clinical course, and discharge diagnoses. We assess the utility of CDrugRed by benchmarking several state-of-the-art large language models (LLMs) on the discharge medication recommendation task. Experimental results show that while supervised fine-tuning improves model performance, there remains substantial room for improvement, with the best model achieving the F1 score of 0.5648 and Jaccard score of 0.4477. This result highlights the complexity of the clinical drug recommendation task and establishes CDrugRed as a challenging and valuable resource for developing more robust and accurate drug recommendation systems. The dataset is publicly available to the research community under the data usage agreements at https://github.com/DUTIR-BioNLP/CDrugRed.

Method: Proposes Self-Rewarding PPO that uses a reward function as the log policy ratio between SFT model and pretrained base model, enabling on-policy fine-tuning without human preference data.

Result: Empirical evaluation shows Self-Rewarding PPO consistently outperforms traditional SFT methods across various NLP tasks, especially in scarce data scenarios.

Conclusion: The approach effectively addresses SFT limitations by improving generalization, data efficiency, and robustness through on-policy fine-tuning with self-rewarding mechanism.

Abstract: Supervised fine-tuning (SFT) has emerged as a crucial method for aligning large language models (LLMs) with human-annotated demonstrations. However, SFT, being an off-policy approach similar to behavior cloning, often struggles with overfitting and poor out-of-domain generalization, especially in limited-data scenarios. To address these limitations, we propose Self-Rewarding PPO, a novel fine-tuning method that leverages on-policy techniques to enhance generalization performance. Our approach combines the strengths of SFT and proximal policy optimization (PPO) to achieve more effective alignment from demonstration data. At its core is a reward function designed as the log policy ratio between the SFT model and the pretrained base model. This function serves as an implicit reward signal, using the pretrained policy as a baseline and the SFT policy as a target. By doing so, it enables on-policy fine-tuning without relying on human preference annotations. The integration of this self-rewarding mechanism with PPO addresses key limitations of SFT, improving generalization, data efficiency, and robustness. Our empirical evaluation across a range of natural language processing tasks demonstrates that Self-Rewarding PPO consistently outperforms traditional SFT methods. The results highlight the effectiveness of our approach in aligning LLMs using demonstration data, particularly in scenarios where high-quality annotated data is scarce.

Method: Proposed a novel faithfulness annotation framework with an intermediate “Out-Dependent” category for cases requiring external knowledge verification, and constructed the VeriGray benchmark using this framework.

Result: Even state-of-the-art LLMs like GPT-5 exhibit ~6% hallucinations in summarization, with ~8% of sentences falling into the Out-Dependent category, highlighting annotation ambiguity challenges.

Conclusion: The VeriGray benchmark poses significant challenges to baseline methods, indicating substantial room for improvement in unfaithfulness detection and the importance of resolving annotation ambiguity.

Abstract: Ensuring that Large Language Models (LLMs) generate summaries faithful to a given source document is essential for real-world applications. While prior research has explored LLM faithfulness, existing benchmarks suffer from annotation ambiguity, primarily due to the ill-defined boundary of permissible external knowledge in generated outputs. For instance, common sense is often incorporated into responses and labeled as “faithful”, yet the acceptable extent of such knowledge remains unspecified, leading to inconsistent annotations. To address this issue, we propose a novel faithfulness annotation framework, which introduces an intermediate category, Out-Dependent, to classify cases where external knowledge is required for verification. Using this framework, we construct VeriGray (Verification with the Gray Zone) – a new unfaithfulness detection benchmark in summarization. Statistics reveal that even SOTA LLMs, such as GPT-5, exhibit hallucinations ($\sim 6%$ of sentences) in summarization tasks. Moreover, a substantial proportion ($\sim 8%$ on average of models) of generated sentences fall into the Out-Dependent category, underscoring the importance of resolving annotation ambiguity in unfaithfulness detection benchmarks. Experiments demonstrate that our benchmark poses significant challenges to multiple baseline methods, indicating considerable room for future improvement.

Method: Encodes speech waveforms into continuous latent representations and models them autoregressively using an energy distance objective that measures distributional gaps between simulated and target samples.

Result: SLED achieves strong performance in both zero-shot and streaming speech synthesis, demonstrating its effectiveness for general-purpose speech language modeling.

Conclusion: SLED simplifies the speech modeling pipeline while preserving speech information richness and maintaining inference efficiency, showing potential for broader applications.

Abstract: We introduce SLED, an alternative approach to speech language modeling by encoding speech waveforms into sequences of continuous latent representations and modeling them autoregressively using an energy distance objective. The energy distance offers an analytical measure of the distributional gap by contrasting simulated and target samples, enabling efficient training to capture the underlying continuous autoregressive distribution. By bypassing reliance on residual vector quantization, SLED avoids discretization errors and eliminates the need for the complicated hierarchical architectures common in existing speech language models. It simplifies the overall modeling pipeline while preserving the richness of speech information and maintaining inference efficiency. Empirical results demonstrate that SLED achieves strong performance in both zero-shot and streaming speech synthesis, showing its potential for broader applications in general-purpose speech language models.

Method: Introduces a taxonomy covering two directions: LLM for TAG (enriching graph tasks) and TAG for LLM (improving LLM reasoning). Categorizes orchestration strategies into sequential, parallel, and multi-module frameworks, and discusses TAG-specific pretraining, prompting, and parameter-efficient fine-tuning.

Result: The survey summarizes empirical insights, curates available datasets, and demonstrates diverse applications in recommendation systems, biomedical analysis, and knowledge-intensive question answering.

Conclusion: The integration of LLMs and TAGs represents a promising research direction that combines the strengths of both approaches, with open challenges remaining in areas like orchestration strategies and domain-specific applications.

Abstract: Large Language Models (LLMs) have achieved remarkable success in natural language processing through strong semantic understanding and generation. However, their black-box nature limits structured and multi-hop reasoning. In contrast, Text-Attributed Graphs (TAGs) provide explicit relational structures enriched with textual context, yet often lack semantic depth. Recent research shows that combining LLMs and TAGs yields complementary benefits: enhancing TAG representation learning and improving the reasoning and interpretability of LLMs. This survey provides the first systematic review of LLM–TAG integration from an orchestration perspective. We introduce a novel taxonomy covering two fundamental directions: LLM for TAG, where LLMs enrich graph-based tasks, and TAG for LLM, where structured graphs improve LLM reasoning. We categorize orchestration strategies into sequential, parallel, and multi-module frameworks, and discuss advances in TAG-specific pretraining, prompting, and parameter-efficient fine-tuning. Beyond methodology, we summarize empirical insights, curate available datasets, and highlight diverse applications across recommendation systems, biomedical analysis, and knowledge-intensive question answering. Finally, we outline open challenges and promising research directions, aiming to guide future work at the intersection of language and graph learning.

Method: A framework simulating multi-agent interactions through chatroom-style conversations, analyzed across five linguistic dimensions to examine emergent social cognitive biases. Experiments involved eight representative LLMs across three families.

Result: LLMs do not faithfully reproduce genuine human behavior but reflect overly idealized versions shaped by social desirability bias, showing social role bias, primacy effect, and positivity bias.

Conclusion: LLMs create “Utopian” societies lacking real human complexity, calling for more socially grounded models that capture the diversity of human social behavior.

Abstract: Reliable simulation of human behavior is essential for explaining, predicting, and intervening in our society. Recent advances in large language models (LLMs) have shown promise in emulating human behaviors, interactions, and decision-making, offering a powerful new lens for social science studies. However, the extent to which LLMs diverge from authentic human behavior in social contexts remains underexplored, posing risks of misinterpretation in scientific studies and unintended consequences in real-world applications. Here, we introduce a systematic framework for analyzing LLMs’ behavior in social simulation. Our approach simulates multi-agent interactions through chatroom-style conversations and analyzes them across five linguistic dimensions, providing a simple yet effective method to examine emergent social cognitive biases. We conduct extensive experiments involving eight representative LLMs across three families. Our findings reveal that LLMs do not faithfully reproduce genuine human behavior but instead reflect overly idealized versions of it, shaped by the social desirability bias. In particular, LLMs show social role bias, primacy effect, and positivity bias, resulting in “Utopian” societies that lack the complexity and variability of real human interactions. These findings call for more socially grounded LLMs that capture the diversity of human social behavior.

Method: Created benchmark using seven diverse datasets from native Estonian sources (no machine translation), evaluated 6 base models and 26 instruction-tuned models using both human evaluation and LLM-as-a-judge methods.

Result: Human evaluation showed moderate to high correlation with benchmark evaluations. Claude 3.7 Sonnet demonstrated strong alignment with human ratings as an LLM judge.

Conclusion: Top-performing LLMs can effectively support the evaluation of Estonian-language models, with Claude 3.7 Sonnet showing strong performance as an automated evaluator.

Abstract: The availability of LLM benchmarks for the Estonian language is limited, and a comprehensive evaluation comparing the performance of different LLMs on Estonian tasks has yet to be conducted. We introduce a new benchmark for evaluating LLMs in Estonian, based on seven diverse datasets. These datasets assess general and domain-specific knowledge, understanding of Estonian grammar and vocabulary, summarization abilities, contextual comprehension, and more. The datasets are all generated from native Estonian sources without using machine translation. We compare the performance of base models, instruction-tuned open-source models, and commercial models. Our evaluation includes 6 base models and 26 instruction-tuned models. To assess the results, we employ both human evaluation and LLM-as-a-judge methods. Human evaluation scores showed moderate to high correlation with benchmark evaluations, depending on the dataset. Claude 3.7 Sonnet, used as an LLM judge, demonstrated strong alignment with human ratings, indicating that top-performing LLMs can effectively support the evaluation of Estonian-language models.

Method: Uses state-of-the-art natural language processing techniques combined with sociological insights to analyze the MIGR-TWIT corpus of far-right tweets in English and French.

Result: The methodology aims to uncover patterns of discourse surrounding migration, hate speech, and persuasion techniques employed by right and far-right actors.

Conclusion: The integrated linguistic, sociological, and computational approach provides cross-disciplinary insights into societal dynamics and helps understand contemporary challenges posed by right-wing extremism on social media.

Abstract: The rise of right-wing populism in Europe has brought to the forefront the significance of analysing social media discourse to understand the dissemination of extremist ideologies and their impact on political outcomes. Twitter, as a platform for interaction and mobilisation, provides a unique window into the everyday communication of far-right supporters. In this paper, we propose a methodology that uses state-of-the-art natural language processing techniques with sociological insights to analyse the MIGR-TWIT corpus of far-right tweets in English and French. We aim to uncover patterns of discourse surrounding migration, hate speech, and persuasion techniques employed by right and far-right actors. By integrating linguistic, sociological, and computational approaches, we seek to offer cross-disciplinary insights into societal dynamics and contribute to a better understanding of contemporary challenges posed by right-wing extremism on social media platforms.

Method: Created clinical taxonomy (32 chief complaints, 6 caller identities) and six-phase call protocol. Developed AutoGen-based multi-agent system with Caller and Dispatcher Agents using fact commons for clinical plausibility. Evaluated with physician assessment of 100 cases and automated linguistic analysis.

Result: High performance with excellent Dispatch Effectiveness (94% correct agent contact) and Guidance Efficacy (91% advice provision). Automated metrics showed neutral affective profile (73.7% neutral sentiment), high readability (Flesch 80.9), and consistently polite style.

Conclusion: The taxonomy-grounded multi-agent system successfully simulates diverse, clinically plausible dispatch scenarios, supporting use for dispatcher training, protocol evaluation, and as foundation for real-time decision support in emergency response.

Abstract: Objective: Emergency medical dispatch (EMD) is a high-stakes process challenged by caller distress, ambiguity, and cognitive load. Large Language Models (LLMs) and Multi-Agent Systems (MAS) offer opportunities to augment dispatchers. This study aimed to develop and evaluate a taxonomy-grounded, LLM-powered multi-agent system for simulating realistic EMD scenarios. Methods: We constructed a clinical taxonomy (32 chief complaints, 6 caller identities from MIMIC-III) and a six-phase call protocol. Using this framework, we developed an AutoGen-based MAS with Caller and Dispatcher Agents. The system grounds interactions in a fact commons to ensure clinical plausibility and mitigate misinformation. We used a hybrid evaluation framework: four physicians assessed 100 simulated cases for “Guidance Efficacy” and “Dispatch Effectiveness,” supplemented by automated linguistic analysis (sentiment, readability, politeness). Results: Human evaluation, with substantial inter-rater agreement (Gwe’s AC1 > 0.70), confirmed the system’s high performance. It demonstrated excellent Dispatch Effectiveness (e.g., 94 % contacting the correct potential other agents) and Guidance Efficacy (advice provided in 91 % of cases), both rated highly by physicians. Algorithmic metrics corroborated these findings, indicating a predominantly neutral affective profile (73.7 % neutral sentiment; 90.4 % neutral emotion), high readability (Flesch 80.9), and a consistently polite style (60.0 % polite; 0 % impolite). Conclusion: Our taxonomy-grounded MAS simulates diverse, clinically plausible dispatch scenarios with high fidelity. Findings support its use for dispatcher training, protocol evaluation, and as a foundation for real-time decision support. This work outlines a pathway for safely integrating advanced AI agents into emergency response workflows.

Method: The authors introduce correlation dimension, a fractal-geometric measure of self-similarity, to quantify the hierarchical recurrence structure of language and bridge local and global properties in a unified framework.

Result: Correlation dimension reveals three distinct phases during pretraining, reflects context-dependent complexity, indicates hallucination tendencies, and reliably detects multiple forms of degeneration in generated text. The method is computationally efficient, robust to model quantization, and applicable across autoregressive architectures.

Conclusion: Correlation dimension provides fresh insight into the generative dynamics of LLMs and offers a more comprehensive evaluation framework that captures both local and global structural properties of language.

Abstract: Large language models (LLMs) have achieved remarkable progress in natural language generation, yet they continue to display puzzling behaviors – such as repetition and incoherence – even when exhibiting low perplexity. This highlights a key limitation of conventional evaluation metrics, which emphasize local prediction accuracy while overlooking long-range structural complexity. We introduce correlation dimension, a fractal-geometric measure of self-similarity, to quantify the epistemological complexity of text as perceived by a language model. This measure captures the hierarchical recurrence structure of language, bridging local and global properties in a unified framework. Through extensive experiments, we show that correlation dimension (1) reveals three distinct phases during pretraining, (2) reflects context-dependent complexity, (3) indicates a model’s tendency toward hallucination, and (4) reliably detects multiple forms of degeneration in generated text. The method is computationally efficient, robust to model quantization (down to 4-bit precision), broadly applicable across autoregressive architectures (e.g., Transformer and Mamba), and provides fresh insight into the generative dynamics of LLMs.

Method: Proposed Permuted Block-Sparse Attention (PBS-Attn) that leverages attention permutation properties to increase block-level sparsity, implemented with custom permuted-FlashAttention kernels.

Result: PBS-Attn consistently outperforms existing block-sparse attention methods in accuracy and closely matches full attention baseline, achieving 2.75x end-to-end speedup in long-context prefilling.

Conclusion: PBS-Attn is a practical and effective solution for scaling LLM context lengths, offering significant computational efficiency gains while maintaining model accuracy.

Abstract: Scaling the context length of large language models (LLMs) offers significant benefits but is computationally expensive. This expense stems primarily from the self-attention mechanism, whose $O(N^2)$ complexity with respect to sequence length presents a major bottleneck for both memory and latency. Fortunately, the attention matrix is often sparse, particularly for long sequences, suggesting an opportunity for optimization. Block-sparse attention has emerged as a promising solution that partitions sequences into blocks and skips computation for a subset of these blocks. However, the effectiveness of this method is highly dependent on the underlying attention patterns, which can lead to sub-optimal block-level sparsity. For instance, important key tokens for queries within a single block may be scattered across numerous other blocks, leading to computational redundancy. In this work, we propose Permuted Block-Sparse Attention (\textbf{PBS-Attn}), a plug-and-play method that leverages the permutation properties of attention to increase block-level sparsity and enhance the computational efficiency of LLM prefilling. We conduct comprehensive experiments on challenging real-world long-context datasets, demonstrating that PBS-Attn consistently outperforms existing block-sparse attention methods in model accuracy and closely matches the full attention baseline. Powered by our custom permuted-FlashAttention kernels, PBS-Attn achieves an end-to-end speedup of up to $2.75\times$ in long-context prefilling, confirming its practical viability. Code available at https://github.com/xinghaow99/pbs-attn

Method: PARL method encodes actions, states, and rewards in prompts to enable LLMs to learn through trial-and-error interaction without fine-tuning.

Result: PARL matches or outperforms traditional RL agents in simple environments but shows limitations in tasks requiring complex mathematical operations or state/action decoding.

Conclusion: LLMs can function as effective RL agents in certain structured tasks through prompting alone, though their performance is constrained by mathematical reasoning capabilities.

Abstract: Large language models (LLMs) have demonstrated high performance on tasks expressed in natural language, particularly in zero- or few-shot settings. These are typically framed as supervised (e.g., classification) or unsupervised (e.g., clustering) problems. However, limited work evaluates LLMs as agents in reinforcement learning (RL) tasks (e.g., playing games), where learning occurs through interaction with an environment and a reward system. While prior work focused on representing tasks that rely on a language representation, we study structured, non-linguistic reasoning - such as interpreting positions in a grid world. We therefore introduce PARL (Prompt-based Agent for Reinforcement Learning), a method that uses LLMs as RL agents through prompting, without any fine-tuning. PARL encodes actions, states, and rewards in the prompt, enabling the model to learn through trial-and-error interaction. We evaluate PARL on three standard RL tasks that do not entirely rely on natural language. We show that it can match or outperform traditional RL agents in simple environments by leveraging pretrained knowledge. However, we identify performance limitations in tasks that require complex mathematical operations or decoding states and actions.

Method: Inject semantic-similarity penalty using NLI model finetuned on partial prefixes, with importance reweighting and control variates for debiasing and variance reduction.

Result: Matches or surpasses baselines across four QA benchmarks while covering more semantic clusters with the same number of samples.

Conclusion: The modular framework serves as a drop-in enhancement for uncertainty estimation in risk-sensitive LLM deployments.

Abstract: Accurately estimating semantic aleatoric and epistemic uncertainties in large language models (LLMs) is particularly challenging in free-form question answering (QA), where obtaining stable estimates often requires many expensive generations. We introduce a diversity-steered sampler that discourages semantically redundant outputs during decoding, covers both autoregressive and masked diffusion paradigms, and yields substantial sample-efficiency gains. The key idea is to inject a continuous semantic-similarity penalty into the model’s proposal distribution using a natural language inference (NLI) model lightly finetuned on partial prefixes or intermediate diffusion states. We debias downstream uncertainty estimates with importance reweighting and shrink their variance with control variates. Across four QA benchmarks, our method matches or surpasses baselines while covering more semantic clusters with the same number of samples. Being modular and requiring no gradient access to the base LLM, the framework promises to serve as a drop-in enhancement for uncertainty estimation in risk-sensitive model deployments.

Method: Analyzed British National Corpus to quantify word collapse, evaluated BERT’s disambiguation ability, and conducted probing experiments comparing BERT variants trained on clean vs. typoglycemic Wikipedia text.

Result: Performance degradation from scrambling is smaller than expected, with relatively few English words collapsing under typoglycemia and collapsed words occurring in distinct contexts that make disambiguation trivial.

Conclusion: NLP model robustness to typoglycemia stems from limited word collapse in English and strong contextual disambiguation, rather than sophisticated internal processing mechanisms.

Abstract: Research in linguistics has shown that humans can read words with internally scrambled letters, a phenomenon recently dubbed typoglycemia. Some specific NLP models have recently been proposed that similarly demonstrate robustness to such distortions by ignoring the internal order of characters by design. This raises a fundamental question: how can models perform well when many distinct words (e.g., form and from) collapse into identical representations under typoglycemia? Our work, focusing exclusively on the English language, seeks to shed light on the underlying aspects responsible for this robustness. We hypothesize that the main reasons have to do with the fact that (i) relatively few English words collapse under typoglycemia, and that (ii) collapsed words tend to occur in contexts so distinct that disambiguation becomes trivial. In our analysis, we (i) analyze the British National Corpus to quantify word collapse and ambiguity under typoglycemia, (ii) evaluate BERT’s ability to disambiguate collapsing forms, and (iii) conduct a probing experiment by comparing variants of BERT trained from scratch on clean versus typoglycemic Wikipedia text; our results reveal that the performance degradation caused by scrambling is smaller than expected.

Method: Threefold evaluation: (1) automatic metrics (Preservation of Intent, Responsiveness, Adaptability), (2) LLM-as-a-judge assessment, (3) manual expert evaluation of semantic, spatial, sequential, and responsive aspects.

Result: LLMs maintain strong sequential consistency and semantic stability, with spatial deviations larger for shorter trips but decreasing with longer ones. Disruption-handling ability declines as plan length increases.

Conclusion: TripTide establishes a benchmark for evaluating adaptability, personalization, and resilience in LLM-based travel planning under real-world uncertainty, revealing limitations in LLM robustness for longer plans.

Abstract: Recent efforts like TripCraft and TravelPlanner have advanced the use of Large Language Models ( LLMs) for personalized, constraint aware travel itinerary generation. Yet, real travel often faces disruptions. To address this, we present TripTide, the first benchmark evaluating LLM’s ability to revise itineraries under realistic disruptions. TripTide models key dimensions such as disruption severity and traveler tolerance, enabling nuanced assessment of LLM adaptability to events like flight cancellations, weather closures, or overbooked attractions. We conduct a threefold evaluation. First, we introduce automatic metrics including Preservation of Intent (how well the revised plan maintains feasibility and goals), Responsiveness (promptness and appropriateness of disruption handling), and Adaptability (semantic, spatial, and sequential divergence between original and revised plans). Second, we apply an LLM-as-a-judge approach to automatically assess revision quality. Third, we perform manual expert evaluation to verify whether revisions preserve semantic, spatial, sequential, and responsive aspects. Our experiments show that LLMs maintain strong sequential consistency and semantic stability, while spatial deviations are larger for shorter trips but decrease with longer ones, indicating that extended plans encourage better geographic coherence. However, disruption-handling ability declines as plan length increases, highlighting limits in LLM robustness. TripTide establishes a benchmark for evaluating adaptability, personalization, and resilience in LLM-based travel planning under real-world uncertainty.

Method: Compared conventional single-turn training with three multi-turn strategies on reasoning tasks.

Result: Single-turn trained models generalize effectively to both single- and multi-turn evaluations, while multi-turn trained models show significant degradation in single-turn reasoning performance.

Conclusion: For tasks with complete information, robust single-turn training is more effective and reliable than multi-turn training, which provides limited benefits and can degrade reasoning capabilities.

Abstract: The reasoning capabilities of Large Language Models (LLMs) are typically developed through the single-turn reinforcement learning, whereas real-world applications often involve multi-turn interactions with human feedback, leading to a potential mismatch between training and deployment conditions. In this work, we study whether multi-turn training with human feedback is necessary for reasoning tasks. We compare conventional single-turn training with three multi-turn strategies and reach contrary conclusions to previous research. We find that models trained in a single-turn setting generalize effectively to both single- and multi-turn evaluations, while models trained with multi-turn strategies exhibit a significant degradation in single-turn reasoning performance. These results suggest that for tasks with complete information, robust single-turn training remains more effective and reliable, as multi-turn training with basic feedback provides limited benefits and can even degrade reasoning capabilities.

Method: Created a manually written QA benchmark inspired by Swedish radio programs featuring public figures and major sports events, with English translations for cross-lingual analysis.

Result: Smaller models with stronger Swedish coverage performed comparably to a three times larger multilingual model on Sweden-related facts. Continued pre-training on Swedish improved factual knowledge but caused some forgetting of previously known information.

Conclusion: The dataset serves as a valuable diagnostic tool for studying language adaptation and knowledge retention in multilingual models during language-specific training.

Abstract: Many Swedish benchmarks are translated US-centric benchmarks, and therefore not suitable for testing knowledge that is particularly relevant, or even specific, to Sweden. We therefore introduce a manually written question-answering benchmark specifically targeted to Sweden-related personalities and events, many of which receive very limited coverage in international media. Our annotators drew inspiration from a popular radio program featuring public figures from culture and media, as well as major sports events in Sweden. The dataset can be used to measure factual recall across models of varying sizes and degrees of Swedish coverage, and allows to probe cross-lingual factual consistency as to contains English translations. Using the dataset, we find that smaller models with stronger Swedish coverage perform comparably to a three times larger multilingual model in recalling Sweden-related facts. We also observe that continued pre-training on Swedish generally improves factual knowledge but also leads to forgetting of a part of the previously known information. These results demonstrate the dataset’s potential as a diagnostic tool for studying language adaptation and knowledge retention in multilingual models and during language adaptation.

Method: Trained from scratch on 312GB of Turkish text (mC4, OSCAR23, Wikipedia) using RoBERTa architecture, released in base and large configurations.

Result: SindBERT performs competitively with existing Turkish and multilingual models, with large variant achieving best scores in 2 of 4 tasks but showing no consistent scaling advantage. Corpus quality can outweigh data volume.

Conclusion: SindBERT provides an open resource for Turkish NLP and demonstrates the limits of scaling and importance of corpus composition for morphologically rich languages.

Abstract: Transformer models have revolutionized NLP, yet many morphologically rich languages remain underrepresented in large-scale pre-training efforts. With SindBERT, we set out to chart the seas of Turkish NLP, providing the first large-scale RoBERTa-based encoder for Turkish. Trained from scratch on 312 GB of Turkish text (mC4, OSCAR23, Wikipedia), SindBERT is released in both base and large configurations, representing the first large-scale encoder-only language model available for Turkish. We evaluate SindBERT on part-of-speech tagging, named entity recognition, offensive language detection, and the TurBLiMP linguistic acceptability benchmark. Our results show that SindBERT performs competitively with existing Turkish and multilingual models, with the large variant achieving the best scores in two of four tasks but showing no consistent scaling advantage overall. This flat scaling trend, also observed for XLM-R and EuroBERT, suggests that current Turkish benchmarks may already be saturated. At the same time, comparisons with smaller but more curated models such as BERTurk highlight that corpus quality and diversity can outweigh sheer data volume. Taken together, SindBERT contributes both as an openly released resource for Turkish NLP and as an empirical case study on the limits of scaling and the central role of corpus composition in morphologically rich languages. The SindBERT models are released under the MIT license and made available in both fairseq and Huggingface formats.

Method: Train RoBERTa-based encoder family (base and large) from scratch on 49.1GB of deduplicated Hebrew web text and Wikipedia using Hebrew-specific byte-level BPE vocabulary.

Result: HalleluBERT outperforms both monolingual and multilingual baselines on NER and sentiment classification benchmarks, setting new state-of-the-art for Hebrew.

Conclusion: HalleluBERT demonstrates the benefits of fully converged monolingual pretraining for Hebrew NLP tasks.

Abstract: Transformer-based models have advanced NLP, yet Hebrew still lacks a large-scale RoBERTa encoder which is extensively trained. Existing models such as HeBERT, AlephBERT, and HeRo are limited by corpus size, vocabulary, or training depth. We present HalleluBERT, a RoBERTa-based encoder family (base and large) trained from scratch on 49.1~GB of deduplicated Hebrew web text and Wikipedia with a Hebrew-specific byte-level BPE vocabulary. Evaluated on NER and sentiment classification benchmarks, HalleluBERT outperforms both monolingual and multilingual baselines. HalleluBERT sets a new state of the art for Hebrew and highlights the benefits of fully converged monolingual pretraining.

Method: The authors introduced a descriptive caption dataset and proposed comprehensive evaluation methods to assess VLMs in human activity recognition, conducting comparative experiments with state-of-the-art deep learning models.

Result: VLMs achieved comparable performance to traditional deep learning models and in some cases even surpassed conventional approaches in terms of accuracy.

Conclusion: This work establishes a benchmark for VLM evaluation in healthcare and opens new possibilities for integrating VLMs into intelligent healthcare systems.

Abstract: As generative AI continues to evolve, Vision Language Models (VLMs) have emerged as promising tools in various healthcare applications. One area that remains relatively underexplored is their use in human activity recognition (HAR) for remote health monitoring. VLMs offer notable strengths, including greater flexibility and the ability to overcome some of the constraints of traditional deep learning models. However, a key challenge in applying VLMs to HAR lies in the difficulty of evaluating their dynamic and often non-deterministic outputs. To address this gap, we introduce a descriptive caption data set and propose comprehensive evaluation methods to evaluate VLMs in HAR. Through comparative experiments with state-of-the-art deep learning models, our findings demonstrate that VLMs achieve comparable performance and, in some cases, even surpass conventional approaches in terms of accuracy. This work contributes a strong benchmark and opens new possibilities for the integration of VLMs into intelligent healthcare systems.

Method: Proposed a utility-based annotation schema that quantifies both positive contributions of relevant passages and negative impacts of distracting ones. Developed UDCG (Utility and Distraction-aware Cumulative Gain) with LLM-oriented positional discount to optimize correlation with end-to-end answer accuracy.

Result: Experiments on five datasets and six LLMs show UDCG improves correlation with RAG performance by up to 36% compared to traditional IR metrics like nDCG, MAP, and MRR.

Conclusion: UDCG provides a critical step toward aligning IR evaluation with LLM consumers and enables more reliable assessment of RAG components by addressing the fundamental misalignments between human-oriented IR metrics and machine consumption patterns.

Abstract: Traditional Information Retrieval (IR) metrics, such as nDCG, MAP, and MRR, assume that human users sequentially examine documents with diminishing attention to lower ranks. This assumption breaks down in Retrieval Augmented Generation (RAG) systems, where search results are consumed by Large Language Models (LLMs), which, unlike humans, process all retrieved documents as a whole rather than sequentially. Additionally, traditional IR metrics do not account for related but irrelevant documents that actively degrade generation quality, rather than merely being ignored. Due to these two major misalignments, namely human vs. machine position discount and human relevance vs. machine utility, classical IR metrics do not accurately predict RAG performance. We introduce a utility-based annotation schema that quantifies both the positive contribution of relevant passages and the negative impact of distracting ones. Building on this foundation, we propose UDCG (Utility and Distraction-aware Cumulative Gain), a metric using an LLM-oriented positional discount to directly optimize the correlation with the end-to-end answer accuracy. Experiments on five datasets and six LLMs demonstrate that UDCG improves correlation by up to 36% compared to traditional metrics. Our work provides a critical step toward aligning IR evaluation with LLM consumers and enables more reliable assessment of RAG components

Method: Integrates multimodal LLMs, IoT devices, and wearables to automatically collect vital signs, accelerometer data, and video clips. Uses anomaly detection modules including fall detection and emergency condition algorithms, plus NLP components for activity/emotion recognition and responding to healthcare inquiries.

Result: Developed a full-fledged prototype that demonstrates the system is implementable and scalable for real-life scenarios, potentially reducing medical workload and healthcare costs.

Conclusion: The system successfully bridges the human-machine interaction gap in remote health monitoring through multimodal LLM integration, enabling real-time patient monitoring with natural language interaction capabilities.

Abstract: With the widespread adoption of wearable devices in our daily lives, the demand and appeal for remote patient monitoring have significantly increased. Most research in this field has concentrated on collecting sensor data, visualizing it, and analyzing it to detect anomalies in specific diseases such as diabetes, heart disease and depression. However, this domain has a notable gap in the aspect of human-machine interaction. This paper proposes REMONI, an autonomous REmote health MONItoring system that integrates multimodal large language models (MLLMs), the Internet of Things (IoT), and wearable devices. The system automatically and continuously collects vital signs, accelerometer data from a special wearable (such as a smartwatch), and visual data in patient video clips collected from cameras. This data is processed by an anomaly detection module, which includes a fall detection model and algorithms to identify and alert caregivers of the patient’s emergency conditions. A distinctive feature of our proposed system is the natural language processing component, developed with MLLMs capable of detecting and recognizing a patient’s activity and emotion while responding to healthcare worker’s inquiries. Additionally, prompt engineering is employed to integrate all patient information seamlessly. As a result, doctors and nurses can access real-time vital signs and the patient’s current state and mood by interacting with an intelligent agent through a user-friendly web application. Our experiments demonstrate that our system is implementable and scalable for real-life scenarios, potentially reducing the workload of medical professionals and healthcare costs. A full-fledged prototype illustrating the functionalities of the system has been developed and being tested to demonstrate the robustness of its various capabilities.

Method: Proposes MRO approach using test-time scaling, reject sampling, and reinforcement learning to optimize token correlations with multiple rewards, plus group step and importance sampling strategies to reduce variance and improve efficiency.

Result: MRO improves reasoning performance and achieves significant sampling speedups while maintaining high performance on reasoning benchmarks.

Conclusion: Enhancing token correlations through MRO effectively addresses DLM limitations in reasoning tasks and enables faster sampling without performance degradation.

Abstract: Recent advances in diffusion language models (DLMs) have presented a promising alternative to traditional autoregressive large language models (LLMs). However, DLMs still lag behind LLMs in reasoning performance, especially as the number of denoising steps decreases. Our analysis reveals that this shortcoming arises primarily from the independent generation of masked tokens across denoising steps, which fails to capture the token correlation. In this paper, we define two types of token correlation: intra-sequence correlation and inter-sequence correlation, and demonstrate that enhancing these correlations improves reasoning performance. To this end, we propose a Multi-Reward Optimization (MRO) approach, which encourages DLMs to consider the token correlation during the denoising process. More specifically, our MRO approach leverages test-time scaling, reject sampling, and reinforcement learning to directly optimize the token correlation with multiple elaborate rewards. Additionally, we introduce group step and importance sampling strategies to mitigate reward variance and enhance sampling efficiency. Through extensive experiments, we demonstrate that MRO not only improves reasoning performance but also achieves significant sampling speedups while maintaining high performance on reasoning benchmarks.

Method: Fine-tune pretrained speech language models to jointly predict fMRI responses from multiple participants simultaneously, creating a multi-participant brain-tuning approach.

Result: 5-fold decrease in fMRI data needed for new participants, up to 50% increase in overall brain alignment, strong generalization to unseen datasets, and improved downstream semantic task performance.

Conclusion: Multi-participant brain-tuning demonstrates bidirectional benefits between neuroscience and AI, bridging the gap between fields while creating more generalizable semantic representations.

Abstract: Pretrained language models are remarkably effective in aligning with human brain responses elicited by natural language stimuli, positioning them as promising model organisms for studying language processing in the brain. However, existing approaches for both estimating and improving this brain alignment are participant-dependent and highly affected by the amount of data available per participant, hindering both generalization to new participants and population-level analyses. In this work, we address these limitations by introducing a scalable, generalizable brain-tuning method, in which we fine-tune pretrained speech language models to jointly predict fMRI responses from multiple participants. We demonstrate that the resulting brain-tuned models exhibit strong individual brain alignment while generalizing across participants. Specifically, our method leads to 1) a 5-fold decrease in the amount of fMRI data needed to predict brain data from new participants, 2) up to a 50% increase in the overall brain alignment, and 3) strong generalization to new unseen datasets. Furthermore, this multi-participant brain-tuning additionally improves downstream performance on semantic tasks, suggesting that training using brain data from multiple participants leads to more generalizable semantic representations. Taken together, these findings demonstrate a bidirectional benefit between neuroscience and AI, helping bridge the gap between the two fields. We make our code and models publicly available at https://github.com/bridge-ai-neuro/multi-brain-tuning.

Method: Compute external context scores and parametric knowledge scores across layers and attention heads in Qwen3-0.6b, then train regression-based classifiers to predict hallucinations.

Result: The method outperforms state-of-the-art LLMs (GPT-5, GPT-4.1) and detection baselines (RAGAS, TruLens, RefChecker), and classifiers generalize to GPT-4.1-mini responses.

Conclusion: Mechanistic signals serve as efficient, generalizable predictors for hallucination detection in RAG systems, enabling proxy-model evaluation.

Abstract: Retrieval-Augmented Generation (RAG) integrates external knowledge to mitigate hallucinations, yet models often generate outputs inconsistent with retrieved content. Accurate hallucination detection requires disentangling the contributions of external context and parametric knowledge, which prior methods typically conflate. We investigate the mechanisms underlying RAG hallucinations and find they arise when later-layer FFN modules disproportionately inject parametric knowledge into the residual stream. To address this, we explore a mechanistic detection approach based on external context scores and parametric knowledge scores. Using Qwen3-0.6b, we compute these scores across layers and attention heads and train regression-based classifiers to predict hallucinations. Our method is evaluated against state-of-the-art LLMs (GPT-5, GPT-4.1) and detection baselines (RAGAS, TruLens, RefChecker). Furthermore, classifiers trained on Qwen3-0.6b signals generalize to GPT-4.1-mini responses, demonstrating the potential of proxy-model evaluation. Our results highlight mechanistic signals as efficient, generalizable predictors for hallucination detection in RAG systems.

Method: 1) Represent documents as categories of question-answer pairs 2) Develop orthogonalization procedure to divide document information into non-overlapping pieces 3) Implement techniques using large pretrained models 4) Use RLVR for self-supervised improvement with consistency constraints

Result: Developed novel information measures, summarization techniques, document extension methods, and a multimodal mathematical framework with self-supervised model improvement.

Conclusion: Category theory provides a powerful mathematical foundation for document structure analysis, enabling new information measures, summarization approaches, and self-supervised model enhancement through consistency constraints.

Abstract: We apply category theory to extract multimodal document structure which leads us to develop information theoretic measures, content summarization and extension, and self-supervised improvement of large pretrained models. We first develop a mathematical representation of a document as a category of question-answer pairs. Second, we develop an orthogonalization procedure to divide the information contained in one or more documents into non-overlapping pieces. The structures extracted in the first and second steps lead us to develop methods to measure and enumerate the information contained in a document. We also build on those steps to develop new summarization techniques, as well as to develop a solution to a new problem viz. exegesis resulting in an extension of the original document. Our question-answer pair methodology enables a novel rate distortion analysis of summarization techniques. We implement our techniques using large pretrained models, and we propose a multimodal extension of our overall mathematical framework. Finally, we develop a novel self-supervised method using RLVR to improve large pretrained models using consistency constraints such as composability and closure under certain operations that stem naturally from our category theoretic framework.

Method: Extend prior analyses on MultiQ dataset by checking if models prefer genealogically related languages when prompt language fidelity is not maintained, and investigate knowledge consistency within vs across genera.

Result: Genus-level effects are present but strongly conditioned by training resource availability, with distinct multilingual strategies observed across LLM families.

Conclusion: LLMs encode aspects of genus-level structure, but training data imbalances remain the primary factor shaping their multilingual performance.

Abstract: Large Language Models (LLMs) display notable variation in multilingual behavior, yet the role of genealogical language structure in shaping this variation remains underexplored. In this paper, we investigate whether LLMs exhibit sensitivity to linguistic genera by extending prior analyses on the MultiQ dataset. We first check if models prefer to switch to genealogically related languages when prompt language fidelity is not maintained. Next, we investigate whether knowledge consistency is better preserved within than across genera. We show that genus-level effects are present but strongly conditioned by training resource availability. We further observe distinct multilingual strategies across LLMs families. Our findings suggest that LLMs encode aspects of genus-level structure, but training data imbalances remain the primary factor shaping their multilingual performance.

Method: Created pragmatics benchmarks for Slovene (SloPragEval and SloPragMega) with 405 multiple-choice questions, established human baselines, and conducted pilot evaluations with various LLMs.

Result: Current models show improved nuanced language understanding but still fail at inferring implied speaker meaning in non-literal utterances, especially culture-specific ones. Significant performance gap exists between proprietary and open-source models.

Conclusion: Benchmarks for nuanced language understanding and cultural knowledge should be carefully designed using native data and validated with human responses, as current LLMs still struggle with pragmatic inference in culturally specific contexts.

Abstract: Large language models are demonstrating increasing capabilities, excelling at benchmarks once considered very difficult. As their capabilities grow, there is a need for more challenging evaluations that go beyond surface-level linguistic competence. Namely, language competence involves not only syntax and semantics but also pragmatics, i.e., understanding situational meaning as shaped by context as well as linguistic and cultural norms. To contribute to this line of research, we introduce SloPragEval and SloPragMega, the first pragmatics understanding benchmarks for Slovene that contain altogether 405 multiple-choice questions. We discuss the difficulties of translation, describe the campaign to establish a human baseline, and report pilot evaluations with LLMs. Our results indicate that current models have greatly improved in understanding nuanced language but may still fail to infer implied speaker meaning in non-literal utterances, especially those that are culture-specific. We also observe a significant gap between proprietary and open-source models. Finally, we argue that benchmarks targeting nuanced language understanding and knowledge of the target culture must be designed with care, preferably constructed from native data, and validated with human responses.

Method: Applied unsupervised anomaly detection using character-level and syllable-level phonotactic features to identify inconsistencies in multilingual Kokborok-Bangla wordlists.

Result: Syllable-aware features significantly outperform character-level baselines, though precision and recall remain modest due to the subtle nature of anomalies. High-recall approach effectively flags entries requiring verification.

Conclusion: The method provides fieldworkers with a systematic approach to improve data quality in low-resourced language documentation by identifying potential transcription errors and borrowings for verification.

Abstract: Lexical data collection in language documentation often contains transcription errors and undocumented borrowings that can mislead linguistic analysis. We present unsupervised anomaly detection methods to identify phonotactic inconsistencies in wordlists, applying them to a multilingual dataset of Kokborok varieties with Bangla. Using character-level and syllable-level phonotactic features, our algorithms identify potential transcription errors and borrowings. While precision and recall remain modest due to the subtle nature of these anomalies, syllable-aware features significantly outperform character-level baselines. The high-recall approach provides fieldworkers with a systematic method to flag entries requiring verification, supporting data quality improvement in low-resourced language documentation.

Method: Propose Reflective Evidence Tuning (RETuning) - a cold-start method that constructs analytical frameworks, organizes/scored evidence for price movements, and enables independent logical reasoning before reinforcement learning.

Result: RETuning successfully unlocks LLMs’ reasoning ability in finance, maintains performance after 6 months and on out-of-distribution stocks, and demonstrates inference-time scaling.

Conclusion: RETuning aligns models with learned analytical frameworks, enabling independent financial reasoning and reducing contextual bias, making LLMs more reliable for stock prediction tasks.

Abstract: Recently, large language models (LLMs) have demonstrated outstanding reasoning capabilities on mathematical and coding tasks. However, their application to financial tasks-especially the most fundamental task of stock movement prediction-remains underexplored. We study a three-class classification problem (up, hold, down) and, by analyzing existing reasoning responses, observe that: (1) LLMs follow analysts’ opinions rather than exhibit a systematic, independent analytical logic (CoTs). (2) LLMs list summaries from different sources without weighing adversarial evidence, yet such counterevidence is crucial for reliable prediction. It shows that the model does not make good use of its reasoning ability to complete the task. To address this, we propose Reflective Evidence Tuning (RETuning), a cold-start method prior to reinforcement learning, to enhance prediction ability. While generating CoT, RETuning encourages dynamically constructing an analytical framework from diverse information sources, organizing and scoring evidence for price up or down based on that framework-rather than on contextual viewpoints-and finally reflecting to derive the prediction. This approach maximally aligns the model with its learned analytical framework, ensuring independent logical reasoning and reducing undue influence from context. We also build a large-scale dataset spanning all of 2024 for 5,123 A-share stocks, with long contexts (32K tokens) and over 200K samples. In addition to price and news, it incorporates analysts’ opinions, quantitative reports, fundamental data, macroeconomic indicators, and similar stocks. Experiments show that RETuning successfully unlocks the model’s reasoning ability in the financial domain. Inference-time scaling still works even after 6 months or on out-of-distribution stocks, since the models gain valuable insights about stock movement prediction.

Method: Information Flow Tracking (IF-Track) uses large language models as probabilistic encoders to quantify information entropy and gain at each reasoning step through fine-grained analyses across diverse tasks.

Result: IF-Track successfully models universal landscape of human reasoning behaviors, captures essential reasoning features, identifies systematic error patterns, characterizes individual differences, and reconciles single- versus dual-process theories.

Conclusion: This approach establishes a quantitative bridge between theory and measurement, offering mechanistic insights into reasoning architecture and discovering alignment between artificial and human cognition.

Abstract: Understanding how information is dynamically accumulated and transformed in human reasoning has long challenged cognitive psychology, philosophy, and artificial intelligence. Existing accounts, from classical logic to probabilistic models, illuminate aspects of output or individual modelling, but do not offer a unified, quantitative description of general human reasoning dynamics. To solve this, we introduce Information Flow Tracking (IF-Track), that uses large language models (LLMs) as probabilistic encoder to quantify information entropy and gain at each reasoning step. Through fine-grained analyses across diverse tasks, our method is the first successfully models the universal landscape of human reasoning behaviors within a single metric space. We show that IF-Track captures essential reasoning features, identifies systematic error patterns, and characterizes individual differences. Applied to discussion of advanced psychological theory, we first reconcile single- versus dual-process theories in IF-Track and discover the alignment of artificial and human cognition and how LLMs reshaping human reasoning process. This approach establishes a quantitative bridge between theory and measurement, offering mechanistic insights into the architecture of reasoning.

Method: EDIT integrates explainability tools (attention maps, integrated gradients) with frequency-based features to detect adversarial perturbations. It then refines inputs using optimal transformation with pre-trained embeddings and model feedback, and includes automated alerting mechanisms for human involvement when needed.

Result: Experiments on BERT and RoBERTa with IMDB, YELP, AGNEWS, and SST2 datasets against seven word substitution attacks show EDIT achieves average F-score of 89.69% and balanced accuracy of 89.70%. It outperforms four state-of-the-art defenses by 1.22x in balanced accuracy and 1.33x in F1-score, while being 83x faster in feature extraction.

Conclusion: EDIT provides robust, interpretable, and efficient protection against standard, zero-day, and adaptive adversarial threats in text classification models, offering both static defenses and adaptive resilience through feature similarity enforcement and input transformation.

Abstract: Transformer-based text classifiers such as BERT, RoBERTa, T5, and GPT have shown strong performance in natural language processing tasks but remain vulnerable to adversarial examples. These vulnerabilities raise significant security concerns, as small input perturbations can cause severe misclassifications. Existing robustness methods often require heavy computation or lack interpretability. This paper presents a unified framework called Explainability-driven Detection, Identification, and Transformation (EDIT) to strengthen inference-time defenses. EDIT integrates explainability tools, including attention maps and integrated gradients, with frequency-based features to automatically detect and identify adversarial perturbations while offering insight into model behavior. After detection, EDIT refines adversarial inputs using an optimal transformation process that leverages pre-trained embeddings and model feedback to replace corrupted tokens. To enhance security assurance, EDIT incorporates automated alerting mechanisms that involve human analysts when necessary. Beyond static defenses, EDIT also provides adaptive resilience by enforcing internal feature similarity and transforming inputs, thereby disrupting the attackers optimization process and limiting the effectiveness of adaptive adversarial attacks. Experiments using BERT and RoBERTa on IMDB, YELP, AGNEWS, and SST2 datasets against seven word substitution attacks demonstrate that EDIT achieves an average Fscore of 89.69 percent and balanced accuracy of 89.70 percent. Compared to four state-of-the-art defenses, EDIT improves balanced accuracy by 1.22 times and F1-score by 1.33 times while being 83 times faster in feature extraction. The framework provides robust, interpretable, and efficient protection against both standard, zero-day, and adaptive adversarial threats in text classification models.

Method: Extended adhocracy+ platform with two AI-supported debate modules and conducted large-scale user study to examine effects and usability.

Result: Findings from the user study on the effects and usability of the AI-supported debate modules are reported.

Conclusion: The extended platform with AI-supported debate modules is available as open-source software to help improve online discussion quality.

Abstract: Online spaces provide individuals with the opportunity to engage in discussions on important topics and make collective decisions, regardless of their geographic location or time zone. However, without adequate support and careful design, such discussions often suffer from a lack of structure and civility in the exchange of opinions. Artificial intelligence (AI) offers a promising avenue for helping both participants and organizers in managing large-scale online participation processes. This paper introduces an extension of adhocracy+, a large-scale open-source participation platform. Our extension features two AI-supported debate modules designed to improve discussion quality and foster participant interaction. In a large-scale user study we examined the effects and usability of both modules. We report our findings in this paper. The extended platform is available at https://github.com/mabehrendt/discuss2.0.

Method: Teacher generates pseudo-labels using ICL, student predicts using only query input with lightweight adapter, enabling disentanglement of demonstration effects.

Result: Student often outperforms teacher, improves generalization, stability, and efficiency across in-domain and out-of-domain tasks.

Conclusion: The method successfully captures demonstration effects in compact form, enabling efficient inference while maintaining composability with new demonstrations.

Abstract: In-context learning (ICL) enables large language models to perform few-shot learning by conditioning on labeled examples in the prompt. Despite its flexibility, ICL suffers from instability – especially as prompt length increases with more demonstrations. To address this, we treat ICL as a source of weak supervision and propose a parameter-efficient method that disentangles demonstration-induced latent shifts from those of the query. An ICL-based teacher generates pseudo-labels on unlabeled queries, while a student predicts them using only the query input, updating a lightweight adapter. This captures demonstration effects in a compact, reusable form, enabling efficient inference while remaining composable with new demonstrations. Although trained on noisy teacher outputs, the student often outperforms its teacher through pseudo-label correction and coverage expansion, consistent with the weak-to-strong generalization effect. Empirically, our method improves generalization, stability, and efficiency across both in-domain and out-of-domain tasks, surpassing standard ICL and prior disentanglement methods.

Method: TPO directly learns from entire preference trees without sampling paired responses, formulates alignment as Preference List Ranking, and uses Adaptive Step Reward to adjust step-level rewards for fine-grained optimization.

Result: TPO consistently outperforms DPO across five public LLMs on four mathematical reasoning datasets, demonstrating superior performance in complex reasoning tasks.

Conclusion: TPO effectively addresses DPO’s limitations by enabling direct learning from preference trees and fine-grained step-level optimization, achieving better performance in mathematical reasoning tasks.

Abstract: In the domain of complex reasoning tasks, such as mathematical reasoning, recent advancements have proposed the use of Direct Preference Optimization (DPO) to suppress output of dispreferred responses, thereby enhancing the long-chain reasoning capabilities of large language models (LLMs). To this end, these studies employed LLMs to generate preference trees via Tree-of-thoughts (ToT) and sample the paired preference responses required by the DPO algorithm. However, the DPO algorithm based on binary preference optimization is unable to learn multiple responses with varying degrees of preference/dispreference that provided by the preference trees, resulting in incomplete preference learning. In this work, we introduce Tree Preference Optimization (TPO), that does not sample paired preference responses from the preference tree; instead, it directly learns from the entire preference tree during the fine-tuning. Specifically, TPO formulates the language model alignment as a Preference List Ranking problem, where the policy can potentially learn more effectively from a ranked preference list of responses given the prompt. In addition, to further assist LLMs in identifying discriminative steps within long-chain reasoning and increase the relative reward margin in the preference list, TPO utilizes Adaptive Step Reward to adjust the reward values of each step in trajectory for performing fine-grained preference optimization. We carry out extensive experiments on mathematical reasoning tasks to evaluate TPO. The experimental results indicate that TPO consistently outperforms DPO across five public large language models on four datasets. Our code is publicly available at https://github.com/MrBlankness/TPO.git.

Method: Proposes Generalized Induction-Head Model (GIM) - a retrieval-based module that identifies similar sequences using exact n-gram matching and fuzzy matching with neural similarity metrics, inspired by induction heads in LLMs.

Result: In language modeling: improves next-token prediction by up to 25% over interpretable baselines, narrowing gap with black-box LLMs. In fMRI: improves neural response prediction by 20% and provides insights into brain language selectivity.

Conclusion: GIM represents a significant step toward uniting interpretability and performance across domains, demonstrating practical applications in both language modeling and neuroscience.

Abstract: While large transformer models excel in predictive performance, their lack of interpretability restricts their usefulness in high-stakes domains. To remedy this, we propose the Generalized Induction-Head Model (GIM), an interpretable model for next-token prediction inspired by the observation of “induction heads” in LLMs. GIM is a retrieval-based module that identifies similar sequences in the input context by combining exact n-gram matching and fuzzy matching based on a neural similarity metric. We evaluate GIM in two settings: language modeling and fMRI response prediction. In language modeling, GIM improves next-token prediction by up to 25%p over interpretable baselines, significantly narrowing the gap with black-box LLMs. In an fMRI setting, GIM improves neural response prediction by 20% and offers insights into the language selectivity of the brain. GIM represents a significant step toward uniting interpretability and performance across domains. The code is available at https://github.com/ejkim47/generalized-induction-head.

Method: Tensor Product Attention (TPA) factorizes queries, keys, and values into contextual low-rank components using tensor decompositions, integrated with Rotary Position Embedding (RoPE).

Result: T6 model with TPA surpasses or matches performance of standard Transformer baselines (MHA, MQA, GQA, MLA) across perplexity and evaluation benchmarks while enabling longer sequence processing under fixed resources.

Conclusion: TPA provides memory and computational efficiency at decoding stage, solving scalability challenges in modern language models while maintaining model quality.

Abstract: Scaling language models to handle longer input sequences typically necessitates large key-value (KV) caches, resulting in substantial memory overhead during inference. In this paper, we propose Tensor Product Attention (TPA), a novel attention mechanism that uses tensor decompositions to represent queries, keys, and values compactly, substantially shrinking the KV cache size at inference time. By factorizing these representations into contextual low-rank components and seamlessly integrating with Rotary Position Embedding (RoPE), TPA achieves improved model quality alongside memory efficiency. Based on TPA, we introduce the Tensor ProducT ATTenTion Transformer (T6), a new model architecture for sequence modeling. Through extensive empirical evaluation on language modeling tasks, we demonstrate that T6 surpasses or matches the performance of standard Transformer baselines including Multi-Head Attention (MHA), Multi-Query Attention (MQA), Grouped-Query Attention (GQA), and Multi-Head Latent Attention (MLA) across various metrics, including perplexity and a range of established evaluation benchmarks. Notably, TPA’s memory efficiency and computational efficiency at decoding stage enables processing longer sequences under fixed resource constraints, addressing a critical scalability challenge in modern language models. Project Page: https://github.com/tensorgi/TPA.

Method: The paper reconstructs the history of misspellings as a scientific problem and examines various mitigation strategies including data augmentation, double step methods, character-order agnostic approaches, and tuple-based methods.

Result: The survey identifies key challenges and opportunities in handling misspellings, analyzes dedicated competitions, and evaluates modern LLM performance against misspellings.

Conclusion: This survey serves as an exhaustive resource for researchers working to mitigate misspelling impacts in the rapidly evolving NLP landscape.

Abstract: This survey provides an overview of the challenges of misspellings in natural language processing (NLP). While often unintentional, misspellings have become ubiquitous in digital communication, especially with the proliferation of Web 2.0, user-generated content, and informal text mediums such as social media, blogs, and forums. Even if humans can generally interpret misspelled text, NLP models frequently struggle to handle it: this causes a decline in performance in common tasks like text classification and machine translation. In this paper, we reconstruct a history of misspellings as a scientific problem. We then discuss the latest advancements to address the challenge of misspellings in NLP. Main strategies to mitigate the effect of misspellings include data augmentation, double step, character-order agnostic, and tuple-based methods, among others. This survey also examines dedicated data challenges and competitions to spur progress in the field. Critical safety and ethical concerns are also examined, for example, the voluntary use of misspellings to inject malicious messages and hate speech on social networks. Furthermore, the survey explores psycholinguistic perspectives on how humans process misspellings, potentially informing innovative computational techniques for text normalization and representation. Finally, the misspelling-related challenges and opportunities associated with modern large language models are also analyzed, including benchmarks, datasets, and performances of the most prominent language models against misspellings. This survey aims to be an exhaustive resource for researchers seeking to mitigate the impact of misspellings in the rapidly evolving landscape of NLP.

Method: Retains original vocabulary while introducing embeddings for frequent n-grams, learned with a separate model during training and stored in off-accelerator memory for minimal inference latency impact.

Result: A 1B-parameter model with SCONE outperforms a 1.9B-parameter baseline across diverse corpora while using only about half the FLOPS and accelerator memory during inference.

Conclusion: SCONE enables effective scaling through n-gram embeddings and separate model scaling while maintaining fixed accelerator usage, demonstrating significant performance improvements with reduced computational requirements.

Abstract: We propose $SCONE$ ($S$calable, $C$ontextualized, $O$ffloaded, $N$-gram $E$mbedding), a new method for extending input embedding layers to enhance language model performance. To avoid increased decoding costs, $SCONE$ retains the original vocabulary while introducing embeddings for a set of frequent n-grams. These embeddings provide contextualized representation for each input token and are learned with a separate model during training. After training, embeddings are precomputed and stored in off-accelerator memory; during inference, querying them has minimal impact on latency due to the low complexity of embedding lookups. $SCONE$ enables two new scaling strategies: increasing the number of n-gram embeddings and scaling the model used to learn them, both while maintaining fixed accelerator usage during inference (in terms of FLOPS and memory). We show that scaling both aspects enables a model with 1B accelerator-resident parameters to outperform a 1.9B-parameter baseline across diverse corpora, while using only about half the FLOPS and accelerator memory during inference.

Method: ECP maps inference-time learning to semantic electromotive force and inference-time reasoning to resistive networks governed by Ohm’s and Faraday’s laws, validated on 70,000 samples across 350 reasoning tasks and 9 advanced LLMs.

Result: ECP achieved ~60% improvement in Pearson correlation compared to conventional inference-time scaling law, explained 15 established prompting strategies, and enabled development of new modular interventions that exceeded median scores of top 80% participants in IOI and IMO competitions.

Conclusion: ECP provides a rigorous circuit-based framework for predicting LLM performance and optimizing modular components, effectively grounding LLM reasoning in electronic-circuit principles.

Abstract: Large language models (LLMs) such as DeepSeek-R1 have achieved remarkable performance across diverse reasoning tasks. To uncover the principles that govern their behaviour, we introduce the Electronic Circuit Principles (ECP), which maps inference-time learning (ITL) onto a semantic electromotive force and inference-time reasoning (ITR) onto a resistive network governed by Ohm’s and Faraday’s laws. This circuit-based modelling yields closed-form predictions of task performance and reveals how modular prompt components interact to shape accuracy. We validated ECP on 70,000 samples spanning 350 reasoning tasks and 9 advanced LLMs, observing a about 60% improvement in Pearson correlation relative to the conventional inference-time scaling law. Moreover, ECP explains the efficacy of 15 established prompting strategies and directs the development of new modular interventions that exceed the median score of the top 80% of participants in both the International Olympiad in Informatics and the International Mathematical Olympiad. By grounding LLM reasoning in electronic-circuit principles, ECP provides a rigorous framework for predicting performance and optimising modular components.

Method: Reframing data cleaning as an anomaly detection problem to dynamically filter noisy content, applied to a corpus constructed from Common Crawl data and existing multilingual sources.

Result: Substantial improvements in data quality, robustness of cleaning pipeline, and downstream performance on multilingual benchmarks, especially for low-resource languages.

Conclusion: The anomaly detection approach to data cleaning effectively enhances multilingual dataset quality and LLM performance across diverse languages.

Abstract: The rapid development of multilingual large language models (LLMs) highlights the need for high-quality, diverse, and well-curated multilingual datasets. In this paper, we introduce DCAD-2000 (Data Cleaning as Anomaly Detection), a large-scale multilingual corpus constructed from newly extracted Common Crawl data and existing multilingual sources. DCAD-2000 covers 2,282 languages, 46.72TB of text, and 8.63 billion documents, spanning 155 high- and medium-resource languages and 159 writing scripts. To overcome the limitations of existing data cleaning approaches, which rely on manually designed heuristic thresholds, we reframe data cleaning as an anomaly detection problem. This dynamic filtering paradigm substantially improves data quality by automatically identifying and removing noisy or anomalous content. By fine-tuning LLMs on DCAD-2000, we demonstrate notable improvements in data quality, robustness of the cleaning pipeline, and downstream performance, particularly for low-resource languages across multiple multilingual benchmarks.

Method: Developed a benchmark that challenges LLMs to enumerate all information they possess on specific topics, evaluating whether they recall excessive, insufficient, or precise amounts of information.

Result: All tested LLMs demonstrate understanding of how much they know about specific topics when given sufficient scale, though different architectures show varying emergence rates of this capability.

Conclusion: Awareness of knowledge may be a generalizable attribute of LLMs, but further research is needed to confirm this potential and fully understand the underlying mechanisms.

Abstract: Large Language Models (LLMs) have emerged as highly capable systems and are increasingly being integrated into various uses. However, the rapid pace of their deployment has outpaced a comprehensive understanding of their internal mechanisms and a delineation of their capabilities and limitations. A desired attribute of an intelligent system is its ability to recognize the scope of its own knowledge. To investigate whether LLMs embody this characteristic, we develop a benchmark designed to challenge these models to enumerate all information they possess on specific topics. This benchmark evaluates whether the models recall excessive, insufficient, or the precise amount of information, thereby indicating their awareness of their own knowledge. Our findings reveal that all tested LLMs, given sufficient scale, demonstrate an understanding of how much they know about specific topics. While different architectures exhibit varying rates of this capability’s emergence, the results suggest that awareness of knowledge may be a generalizable attribute of LLMs. Further research is needed to confirm this potential and fully elucidate the underlying mechanisms.

Method: Developed a universal theoretical framework using bipartite mutual information scaling laws, formulated the L²M condition that bounds model history state scaling, and validated on transformer and state-space models.

Result: Demonstrated that bipartite mutual information captures distinct multi-token interactions and provides a more complete characterization of dependencies needed for accurate long-sequence modeling.

Conclusion: The framework provides a principled foundation for understanding long-context modeling and designing more efficient architectures with stronger long-context capabilities, with applications beyond natural language.

Abstract: We present a universal theoretical framework for understanding long-context language modeling based on a bipartite mutual information scaling law that we rigorously verify in natural language. We demonstrate that bipartite mutual information captures multi-token interactions distinct from and scaling independently of conventional two-point mutual information, and show that this provides a more complete characterization of the dependencies needed for accurately modeling long sequences. Leveraging this scaling law, we formulate the Long-context Language Modeling (L$^2$M) condition, which lower bounds the necessary scaling of a model’s history state – the latent variables responsible for storing past information – for effective long-context modeling. We validate the framework and its predictions on transformer and state-space models. Our work provides a principled foundation to understand long-context modeling and to design more efficient architectures with stronger long-context capabilities, with potential applications beyond natural language.

Method: SoT integrates cognitively inspired reasoning paradigms (Conceptual Chaining, Chunked Symbolism, Expert Lexicons) with linguistic constraints, dynamically selected by a lightweight routing model at test-time.

Result: Across 18 reasoning datasets spanning multiple domains, languages, and modalities, SoT achieves token reductions of up to 84% with minimal accuracy loss, and even improves accuracy in mathematical and multi-hop reasoning tasks.

Conclusion: SoT provides an efficient prompting framework that significantly reduces computational costs while preserving or even enhancing reasoning performance across diverse tasks.

Abstract: Recent advances in large language models (LLMs) have enabled strong reasoning capabilities through Chain-of-Thought (CoT) prompting, which elicits step-by-step problem solving, but often at the cost of excessive verbosity in intermediate outputs, leading to increased computational overhead. We propose Sketch-of-Thought (SoT), a prompting framework that integrates cognitively inspired reasoning paradigms with linguistic constraints to reduce token usage while preserving reasoning accuracy. SoT is designed as a flexible, modular approach and is instantiated with three paradigms–Conceptual Chaining, Chunked Symbolism, and Expert Lexicons–each tailored to distinct reasoning tasks and selected dynamically at test-time by a lightweight routing model. Across 18 reasoning datasets spanning multiple domains, languages, and modalities, SoT achieves token reductions of up to 84% with minimal accuracy loss. In tasks such as mathematical and multi-hop reasoning, it even improves accuracy while shortening outputs.

Method: Developed a principled cross-tokenizer distillation method that enables effective knowledge transfer across fundamentally different tokenizers, including transfer from subword models to byte-level models.

Result: Successfully demonstrated three use cases: effective tokenizer transfer, distilling math-specialized LLMs into small general-purpose models with different tokenizers, and training embedding prediction hypernetworks for training-free tokenizer transfer.

Conclusion: The method unlocks an expanded range of teacher-student pairs for distillation, enabling new ways to adapt and enhance interaction between LLMs with different tokenization schemes.

Abstract: Distillation has shown remarkable success in transferring knowledge from a Large Language Model (LLM) teacher to a student LLM. However, current distillation methods require similar tokenizers between the teacher and the student, restricting their applicability to only a small subset of teacher-student pairs. In this work, we develop a principled cross-tokenizer distillation method to solve this crucial deficiency. Our method is the first to enable effective distillation across fundamentally different tokenizers, while also substantially outperforming prior methods in all other cases. We verify the efficacy of our method on three distinct use cases. First, we show that viewing tokenizer transfer as self-distillation enables unprecedentedly effective transfer across tokenizers, including rapid transfer of subword models to the byte-level. Transferring different models to the same tokenizer also enables ensembling to boost performance. Secondly, we distil a large maths-specialised LLM into a small general-purpose model with a different tokenizer, achieving competitive maths problem-solving performance. Thirdly, we use our method to train state-of-the-art embedding prediction hypernetworks for training-free tokenizer transfer. Our results unlock an expanded range of teacher-student pairs for distillation, enabling new ways to adapt and enhance interaction between LLMs.

Method: Paper-to-Sections-to-QAs decomposition with zero-shot LLM prompting for section classification, QA augmentation, and weighted innovation scoring using confidence scores as weights.

Result: HSPIM outperforms baseline methods in effectiveness, generalization, and interpretability on scientific conference paper datasets.

Conclusion: HSPIM provides an effective training-free framework for measuring paper innovation with improved performance and interpretability through hierarchical decomposition and confidence-weighted scoring.

Abstract: Measuring scientific paper innovation is both important and challenging. Existing content-based methods often overlook the full-paper context, fail to capture the full scope of innovation, and lack generalization. We propose HSPIM, a hierarchical and training-free framework based on large language models (LLMs). It introduces a Paper-to-Sections-to-QAs decomposition to assess innovation. We segment the text by section titles and use zero-shot LLM prompting to implement section classification, question-answering (QA) augmentation, and weighted innovation scoring. The generated QA pair focuses on section-level innovation and serves as additional context to improve the LLM scoring. For each chunk, the LLM outputs a novelty score and a confidence score. We use confidence scores as weights to aggregate novelty scores into a paper-level innovation score. To further improve performance, we propose a two-layer question structure consisting of common and section-specific questions, and apply a genetic algorithm to optimize the question-prompt combinations. Furthermore, under the fine-grained structure of innovation, we extend HSPIM to an HSPIM$^+$ that generates novelty, contribution, and feasibility scores with respective confidence scores. Comprehensive experiments on scientific conference paper datasets show that HSPIM outperforms baseline methods in effectiveness, generalization, and interpretability. Demo code is available at https://github.com/Jasaxion/HSPIM.

Method: BLEUBERI identifies challenging instructions and applies Group Relative Policy Optimization (GRPO) using BLEU directly as the reward function, leveraging existing instruction-following datasets or synthetic data.

Result: BLEUBERI-trained models are competitive with reward model-guided RL across four challenging benchmarks and three base language models. Human evaluation shows comparable quality, with outputs being more factually grounded than competing methods.

Conclusion: String matching-based metrics like BLEU are cheap yet effective proxies for reward models during alignment when high-quality reference outputs are available.

Abstract: Reward models are central to aligning LLMs with human preferences, but they are costly to train, requiring large-scale human-labeled preference data and powerful pretrained LLM backbones. Meanwhile, the increasing availability of high-quality synthetic instruction-following datasets raises the question: can simpler, reference-based metrics serve as viable alternatives to reward models during RL-based alignment? In this paper, we show first that BLEU, a basic string-matching metric, surprisingly matches strong reward models in agreement with human preferences on general instruction-following datasets. Based on this insight, we develop BLEUBERI, a method that first identifies challenging instructions and then applies Group Relative Policy Optimization (GRPO) using BLEU directly as the reward function. We demonstrate that BLEUBERI-trained models are competitive with models trained via reward model-guided RL across four challenging instruction-following benchmarks and three different base language models. A human evaluation further supports that the quality of BLEUBERI model outputs is on par with those from reward model-aligned models. Moreover, BLEUBERI models generate outputs that are more factually grounded than competing methods. Overall, we show that given access to high-quality reference outputs (easily obtained via existing instruction-following datasets or synthetic data generation), string matching-based metrics are cheap yet effective proxies for reward models during alignment. We release our code and data at https://github.com/lilakk/BLEUBERI.

Method: Created a CC-BY-4.0 licensed human-annotated preference dataset spanning diverse real-world LLM applications including STEM, coding, and multilingual scenarios, then used it to train Reward Models.

Result: Trained RMs achieved top performance on RM-Bench (82.4%) and JudgeBench (73.7%), representing ~10% absolute improvement over previous best results, and successfully applied to train Generative RMs and align policy models with RLHF.

Conclusion: HelpSteer3-Preference provides a high-quality, diverse preference dataset that enables significant improvements in reward model performance and supports effective RLHF alignment of language models.

Abstract: Preference datasets are essential for training general-domain, instruction-following language models with Reinforcement Learning from Human Feedback (RLHF). Each subsequent data release raises expectations for future data collection, meaning there is a constant need to advance the quality and diversity of openly available preference data. To address this need, we introduce HelpSteer3-Preference, a permissively licensed (CC-BY-4.0), high-quality, human-annotated preference dataset comprising of over 40,000 samples. These samples span diverse real-world applications of large language models (LLMs), including tasks relating to STEM, coding and multilingual scenarios. Using HelpSteer3-Preference, we train Reward Models (RMs) that achieve top performance on RM-Bench (82.4%) and JudgeBench (73.7%). This represents a substantial improvement (~10% absolute) over the previously best-reported results from existing RMs. We demonstrate HelpSteer3-Preference can also be applied to train Generative RMs and how policy models can be aligned with RLHF using our RMs. Dataset (CC-BY-4.0): https://huggingface.co/datasets/nvidia/HelpSteer3#preference Models (NVIDIA Open Model): https://huggingface.co/collections/nvidia/reward-models-68377c5955575f71fcc7a2a3

Method: Created a comprehensive dataset from law school exams with explicit guidance on expected reasoning approaches (issue spotting, rule recall, rule application), and used an ensemble LLM-as-a-Judge paradigm with human expert validation for evaluation.

Result: Current LLMs struggle significantly with open questions requiring structured multi-step legal reasoning, and the dataset effectively differentiates between models with varying capabilities.

Conclusion: LEXam provides a scalable method to assess legal reasoning quality beyond simple accuracy metrics and demonstrates that model-generated reasoning can be evaluated consistently and accurately, aligning with human expert assessments.

Abstract: Long-form legal reasoning remains a key challenge for large language models (LLMs) in spite of recent advances in test-time scaling. To address this, we introduce \textsc{LEXam}, a novel benchmark derived from 340 law exams spanning 116 law school courses across a range of subjects and degree levels. The dataset comprises 4,886 law exam questions in English and German, including 2,841 long-form, open-ended questions and 2,045 multiple-choice questions. Besides reference answers, the open questions are also accompanied by explicit guidance outlining the expected legal reasoning approach such as issue spotting, rule recall, or rule application. Our evaluation on both open-ended and multiple-choice questions present significant challenges for current LLMs; in particular, they notably struggle with open questions that require structured, multi-step legal reasoning. Moreover, our results underscore the effectiveness of the dataset in differentiating between models with varying capabilities. Deploying an ensemble LLM-as-a-Judge paradigm with rigorous human expert validation, we demonstrate how model-generated reasoning steps can be evaluated consistently and accurately, closely aligning with human expert assessments. Our evaluation setup provides a scalable method to assess legal reasoning quality beyond simple accuracy metrics. We have open-sourced our code on https://github.com/LEXam-Benchmark/LEXam and released our data on https://huggingface.co/datasets/LEXam-Benchmark/LEXam. Project page: https://lexam-benchmark.github.io.

Method: HeteroSpec estimates verification complexity using lightweight entropy-based quantifier, partitions candidates via data-driven stratification policy, and dynamically tunes speculative depth and pruning thresholds through coordinated optimization.

Result: Across five benchmarks and four LLMs, HeteroSpec delivers average 4.24x decoding speedup over state-of-the-art methods like EAGLE-3 while preserving exact output distributions.

Conclusion: HeteroSpec provides a practical direction for improving speculative decoding efficiency without model retraining and remains compatible with other inference optimizations.

Abstract: Autoregressive decoding inherently limits the inference throughput of Large Language Model (LLM) due to its sequential dependency. Speculative decoding mitigates this by verifying multiple predicted tokens in parallel, but its efficiency remains constrained by what we identify as verification heterogeneity – the uneven difficulty of verifying different speculative candidates. In practice, a small subset of high-confidence predictions accounts for most successful verifications, yet existing methods treat all candidates uniformly, leading to redundant computation. We present HeteroSpec, a heterogeneity-adaptive speculative decoding framework that allocates verification effort in proportion to candidate uncertainty. HeteroSpec estimates verification complexity using a lightweight entropy-based quantifier, partitions candidates via a data-driven stratification policy, and dynamically tunes speculative depth and pruning thresholds through coordinated optimization. Across five benchmarks and four LLMs, HeteroSpec delivers an average 4.24$\times$ decoding speedup over state-of-the-art methods such as EAGLE-3, while preserving exact output distributions. Crucially, HeteroSpec requires no model retraining and remains compatible with other inference optimizations, making it a practical direction for improving speculative decoding efficiency.

Method: RPC periodically compresses KV cache by retaining entries with high importance scores computed using a selector window of recently generated queries.

Result: RPC improves generation throughput of QwQ-32B by up to 1.60× with only 1.2% accuracy drop on AIME 2024 benchmark.

Conclusion: Semantic sparsity in reasoning traces can be effectively exploited for compression, offering practical path toward efficient deployment of reasoning LLMs.

Abstract: Recent reasoning-focused language models achieve high accuracy by generating lengthy intermediate reasoning paths before producing final answers. While this approach is effective in solving problems that require logical thinking, long reasoning paths significantly increase memory usage and reduce throughput of token generation, limiting the practical deployment of such models. We propose Reasoning Path Compression (RPC), a training-free method that accelerates inference by leveraging the semantic sparsity of reasoning paths. RPC periodically compresses the KV cache by retaining cache entries that receive high importance score, which are computed using a selector window composed of recently generated queries. Experiments show that RPC improves generation throughput of QwQ-32B by up to 1.60$\times$ compared to the inference with full KV cache, with an accuracy drop of 1.2% on the AIME 2024 benchmark. Our findings demonstrate that semantic sparsity in reasoning traces can be effectively exploited for compression, offering a practical path toward efficient deployment of reasoning LLMs. Our code is available at https://github.com/jiwonsong-dev/ReasoningPathCompression.

Method: Developed overthinking identification metrics based on standard answers, created a systematic method to detect redundant reasoning, built adaptive reasoning length data construction strategy, and introduced innovative braking prompt mechanism for self-termination.

Result: Experiments on mathematical benchmarks (AIME, AMC, MATH500, GSM8K) show up to 60% reduction in token consumption while maintaining comparable accuracy to unconstrained models.

Conclusion: SBT successfully enables models to self-regulate reasoning, significantly reducing computational costs without sacrificing performance, representing an effective approach to address overthinking in large reasoning models.

Abstract: Large reasoning models (LRMs), such as OpenAI o1 and DeepSeek-R1, have significantly enhanced their reasoning capabilities by generating longer chains of thought, demonstrating outstanding performance across a variety of tasks. However, this performance gain comes at the cost of a substantial increase in redundant reasoning during the generation process, leading to high computational overhead and exacerbating the issue of overthinking. Although numerous existing approaches aim to address the problem of overthinking, they often rely on external interventions. In this paper, we propose a novel framework, Self-Braking Tuning (SBT), which tackles overthinking from the perspective of allowing the model to regulate its own reasoning process, thus eliminating the reliance on external control mechanisms. We construct a set of overthinking identification metrics based on standard answers and design a systematic method to detect redundant reasoning. This method accurately identifies unnecessary steps within the reasoning trajectory and generates training signals for learning self-regulation behaviors. Building on this foundation, we develop a complete strategy for constructing data with adaptive reasoning lengths and introduce an innovative braking prompt mechanism that enables the model to naturally learn when to terminate reasoning at an appropriate point. Experiments across mathematical benchmarks (AIME, AMC, MATH500, GSM8K) demonstrate that our method reduces token consumption by up to 60% while maintaining comparable accuracy to unconstrained models.

Method: Used judge-LLM signals as rewards to adversarially tune models that generate text preambles designed to boost downstream performance, creating a pipeline where frozen LLMs combined with these tuned preamble generators achieve higher evaluation scores.

Result: The approach achieved higher LLM-evaluation scores than existing frameworks, was virtually undetectable (unlike methods that directly edit responses), and the effectiveness transferred when replacing both candidate-LLM and judge-LLM with models not used during training.

Conclusion: This reveals critical vulnerabilities in LLM-as-a-judge evaluation systems and demonstrates that human preferences can be effectively reverse engineered through reinforcement learning on upstream preambles, with potential applications beyond adversarial attacks.

Abstract: The capabilities of Large Language Models (LLMs) are routinely evaluated by other LLMs trained to predict human preferences. This framework–known as LLM-as-a-judge–is highly scalable and relatively low cost. However, it is also vulnerable to malicious exploitation, as LLM responses can be tuned to overfit the preferences of the judge. Previous work shows that the answers generated by a candidate-LLM can be edited post hoc to maximise the score assigned to them by a judge-LLM. In this study, we adopt a different approach and use the signal provided by judge-LLMs as a reward to adversarially tune models that generate text preambles designed to boost downstream performance. We find that frozen LLMs pipelined with these models attain higher LLM-evaluation scores than existing frameworks. Crucially, unlike other frameworks which intervene directly on the model’s response, our method is virtually undetectable. We also demonstrate that the effectiveness of the tuned preamble generator transfers when the candidate-LLM and the judge-LLM are replaced with models that are not used during training. These findings raise important questions about the design of more reliable LLM-as-a-judge evaluation settings. They also demonstrate that human preferences can be reverse engineered effectively, by pipelining LLMs to optimise upstream preambles via reinforcement learning–an approach that could find future applications in diverse tasks and domains beyond adversarial attacks.

Method: Created T1 dataset with integrated caching mechanism for short/long-term memory, supporting dynamic replanning decisions like recomputing vs reusing cached results.

Result: T1 enables rigorous evaluation of agents’ tool coordination abilities across diverse domains and serves as a benchmark for LLM performance assessment.

Conclusion: T1 facilitates research on tool use and planning, demonstrating LLMs’ capabilities in complex, tool-dependent scenarios through T1-Agent results.

Abstract: Large Language Models (LLMs) have demonstrated impressive capabilities as intelligent agents capable of solving complex problems. However, effective planning in scenarios involving dependencies between API or tool calls-particularly in multi-turn conversations-remains a significant challenge. To address this, we introduce T1, a tool-augmented, multi-domain, multi-turn conversational dataset specifically designed to capture and manage inter-tool dependencies across diverse domains. T1 enables rigorous evaluation of agents' ability to coordinate tool use across nine distinct domains (4 single domain and 5 multi-domain) with the help of an integrated caching mechanism for both short- and long-term memory, while supporting dynamic replanning-such as deciding whether to recompute or reuse cached results. Beyond facilitating research on tool use and planning, T1 also serves as a benchmark for evaluating the performance of open-weight and proprietary large language models. We present results powered by T1-Agent, highlighting their ability to plan and reason in complex, tool-dependent scenarios.

Method: Proposed a limited domain pre-training setting with out-of-distribution testing, analyzed stateful architectures, conducted component analysis, and empirically validated design principles using Top-1 element and chunk selection architectures.

Result: Found significant degradation in base capabilities for stateful architectures compared to Transformer, identified that architectures need full-sequence arbitrary selection capability to avoid degradation, and validated this principle experimentally.

Conclusion: The paper establishes a key architecture design principle and provides valuable reference for future sequence modeling architecture improvements, showing that full-sequence arbitrary selection capability is essential for maintaining base capabilities.

Abstract: Pre-trained language models represented by the Transformer have been proven to possess strong base capabilities, and the representative self-attention mechanism in the Transformer has become a classic in sequence modeling architectures. Different from the work of proposing sequence modeling architecture to improve the efficiency of attention mechanism, this work focuses on the impact of sequence modeling architectures on base capabilities. Specifically, our concern is: How exactly do sequence modeling architectures affect the base capabilities of pre-trained language models? In this work, we first point out that the mixed domain pre-training setting commonly adopted in existing architecture design works fails to adequately reveal the differences in base capabilities among various architectures. To address this, we propose a limited domain pre-training setting with out-of-distribution testing, which successfully uncovers significant differences in base capabilities among architectures at an early stage. Next, we analyze the base capabilities of stateful sequence modeling architectures, and find that they exhibit significant degradation in base capabilities compared to the Transformer. Then, through a series of architecture component analysis, we summarize a key architecture design principle: A sequence modeling architecture need possess full-sequence arbitrary selection capability to avoid degradation in base capabilities. Finally, we empirically validate this principle using an extremely simple Top-1 element selection architecture and further generalize it to a more practical Top-1 chunk selection architecture. Experimental results demonstrate our proposed sequence modeling architecture design principle and suggest that our work can serve as a valuable reference for future architecture improvements and novel designs.

Method: SEA formulates inference as iterative optimization on an energy function over actions in continuous space, adapting original responses via gradient-based sampling toward the optimal policy.

Result: SEA outperforms second-best baseline with relative improvements of up to 77.51% on AdvBench and 16.36% on MATH.

Conclusion: SEA provides a simple and effective approach for inference-time alignment through continuous space optimization, demonstrating superior performance compared to discrete search methods.

Abstract: Aligning large language models with human feedback at inference time has received increasing attention due to its flexibility. Existing methods rely on generating multiple responses from the base policy for search using a reward model, which can be considered as searching in a discrete response space. However, these methods struggle to explore informative candidates when the base policy is weak or the candidate set is small, resulting in limited effectiveness. In this paper, to address this problem, we propose Simple Energy Adaptation ($\textbf{SEA}$), a simple yet effective algorithm for inference-time alignment. In contrast to expensive search over the discrete space, SEA directly adapts original responses from the base policy toward the optimal one via gradient-based sampling in continuous latent space. Specifically, SEA formulates inference as an iterative optimization procedure on an energy function over actions in the continuous space defined by the optimal policy, enabling simple and effective alignment. For instance, despite its simplicity, SEA outperforms the second-best baseline with a relative improvement of up to $ \textbf{77.51%}$ on AdvBench and $\textbf{16.36%}$ on MATH. Our code is publicly available at https://github.com/yuanyige/sea

Method: The authors propose score normalization for biaffine scoring and conduct experiments on semantic/syntactic dependency parsing across multiple languages using k-hop parsers with stacked BiLSTMs.

Result: Normalizing biaffine scores allows achieving state-of-the-art performance with fewer samples and trainable parameters across various parsing tasks and languages.

Conclusion: Score normalization makes biaffine scoring substantially more efficient and effective, addressing the overparameterization issue while maintaining or improving parsing performance.

Abstract: Dependency parsing is the task of inferring natural language structure, often approached by modeling word interactions via attention through biaffine scoring. This mechanism works like self-attention in Transformers, where scores are calculated for every pair of words in a sentence. However, unlike Transformer attention, biaffine scoring does not use normalization prior to taking the softmax of the scores. In this paper, we provide theoretical evidence and empirical results revealing that a lack of normalization necessarily results in overparameterized parser models, where the extra parameters compensate for the sharp softmax outputs produced by high variance inputs to the biaffine scoring function. We argue that biaffine scoring can be made substantially more efficient by performing score normalization. We conduct experiments on semantic and syntactic dependency parsing in multiple languages, along with latent graph inference on non-linguistic data, using various settings of a $k$-hop parser. We train $N$-layer stacked BiLSTMs and evaluate the parser’s performance with and without normalizing biaffine scores. Normalizing allows us to achieve state-of-the-art performance with fewer samples and trainable parameters. Code: https://github.com/paolo-gajo/EfficientSDP

Method: Introduces MM-VAP, a multimodal predictive turn-taking model that combines speech with visual features (facial expression, head pose, gaze). Uses automatic speech alignment for training and groups turns by duration of silence between them for detailed analysis.

Result: MM-VAP achieves 84% hold/shift prediction accuracy vs 79% for state-of-the-art audio-only model. Outperforms audio-only model across all durations of speaker transitions. Ablation study shows facial expression features contribute most to performance.

Conclusion: Visual cues are vital for accurate turn-taking prediction when interlocutors can see each other. This represents the first comprehensive analysis of multimodal predictive turn-taking models, with implications for future work in human-robot interaction.

Abstract: Turn-taking is richly multimodal. Predictive turn-taking models (PTTMs) facilitate naturalistic human-robot interaction, yet most rely solely on speech. We introduce MM-VAP, a multimodal PTTM which combines speech with visual cues including facial expression, head pose and gaze. We find that it outperforms the state-of-the-art audio-only in videoconferencing interactions (84% vs. 79% hold/shift prediction accuracy). Unlike prior work which aggregates all holds and shifts, we group by duration of silence between turns. This reveals that through the inclusion of visual features, MM-VAP outperforms a state-of-the-art audio-only turn-taking model across all durations of speaker transitions. We conduct a detailed ablation study, which reveals that facial expression features contribute the most to model performance. Thus, our working hypothesis is that when interlocutors can see one another, visual cues are vital for turn-taking and must therefore be included for accurate turn-taking prediction. We additionally validate the suitability of automatic speech alignment for PTTM training using telephone speech. This work represents the first comprehensive analysis of multimodal PTTMs. We discuss implications for future work and make all code publicly available.

Method: Two-stage approach: 1) Cold start to learn iterative reasoning and retrieval, 2) Reinforcement learning with two rewards (answer correctness and relevance-based document verification) to explore retrieval environment.

Result: R3-RAG significantly outperforms baselines and transfers well to different retrievers.

Conclusion: R3-RAG successfully enables LLMs to learn how to reason and retrieve step by step, improving retrieval of comprehensive external knowledge and leading to correct answers.

Abstract: Retrieval-Augmented Generation (RAG) integrates external knowledge with Large Language Models (LLMs) to enhance factual correctness and mitigate hallucination. However, dense retrievers often become the bottleneck of RAG systems due to their limited parameters compared to LLMs and their inability to perform step-by-step reasoning. While prompt-based iterative RAG attempts to address these limitations, it is constrained by human-designed workflows. To address these limitations, we propose $\textbf{R3-RAG}$, which uses $\textbf{R}$einforcement learning to make the LLM learn how to $\textbf{R}$eason and $\textbf{R}$etrieve step by step, thus retrieving comprehensive external knowledge and leading to correct answers. R3-RAG is divided into two stages. We first use cold start to make the model learn the manner of iteratively interleaving reasoning and retrieval. Then we use reinforcement learning to further harness its ability to better explore the external retrieval environment. Specifically, we propose two rewards for R3-RAG: 1) answer correctness for outcome reward, which judges whether the trajectory leads to a correct answer; 2) relevance-based document verification for process reward, encouraging the model to retrieve documents that are relevant to the user question, through which we can let the model learn how to iteratively reason and retrieve relevant documents to get the correct answer. Experimental results show that R3-RAG significantly outperforms baselines and can transfer well to different retrievers. We release R3-RAG at https://github.com/Yuan-Li-FNLP/R3-RAG.

Method: Propose LayerIF framework that isolates each layer’s gradients and computes layer-wise influences by measuring validation loss sensitivity to training examples, deriving data-driven layer importance estimates.

Result: LayerIF produces task-specific layer importance estimates for the same LLM, revealing layer specialization for different evaluation tasks, and leads to consistent performance gains in expert allocation and pruning applications.

Conclusion: LayerIF provides a principled, data-driven approach for layer importance estimation that outperforms model-centric heuristics and enables better downstream task performance through influence-guided allocation strategies.

Abstract: Pretrained Large Language Models (LLMs) achieve strong performance across a wide range of tasks, yet exhibit substantial variability in the various layers' training quality with respect to specific downstream applications, limiting their downstream performance. It is therefore critical to estimate layer-wise training quality in a manner that accounts for both model architecture and training data. However, existing approaches predominantly rely on model-centric heuristics (such as spectral statistics, outlier detection, or uniform allocation) while overlooking the influence of data. To address these limitations, we propose LayerIF, a data-driven framework that leverages Influence Functions to quantify the training quality of individual layers in a principled and task-sensitive manner. By isolating each layer’s gradients and measuring the sensitivity of the validation loss to training examples by computing layer-wise influences, we derive data-driven estimates of layer importance. Notably, our method produces task-specific layer importance estimates for the same LLM, revealing how layers specialize for different test-time evaluation tasks. We demonstrate the utility of our scores by leveraging them for two downstream applications: (a) expert allocation in LoRA-MoE architectures and (b) layer-wise sparsity distribution for LLM pruning. Experiments across multiple LLM architectures demonstrate that our model-agnostic, influence-guided allocation leads to consistent gains in task performance.

Method: zip2zip uses Lempel-Ziv-Welch compression to dynamically merge co-occurring tokens into hypertokens, with dynamic embedding layers and autoregressive language modeling adapted for compressed text sequences.

Result: The method reduces input and output tokens by 15-40% and can be uptrained on existing LLMs in just 10 GPU-hours via parameter-efficient finetuning.

Conclusion: zip2zip enables test-time adaptation where LLMs learn to use hypertokens in unseen contexts, significantly improving tokenization efficiency and reducing computational costs.

Abstract: Tokenization efficiency plays a critical role in the performance and cost of large language models (LLMs), yet most models rely on static tokenizers optimized on general-purpose corpora. These tokenizers’ fixed vocabularies often fail to adapt to domain- or language-specific inputs, leading to longer token sequences and higher computational costs. We introduce zip2zip, a novel method for achieving context-adaptive tokenization in LLMs at inference time. Leveraging an online data compression algorithm (Lempel-Ziv-Welch), zip2zip dynamically expands its active vocabulary at inference time by continuously replacing fragmented token sequences with more compact hypertokens, which it can immediately output during generation. In doing so, the model refines its internal tokenization scheme to match the token distribution of the current context, reducing redundancy and improving representational efficiency. zip2zip consists of three key components: (1) a tokenizer based on Lempel-Ziv-Welch compression that incrementally merges co-occurring tokens into reusable hypertokens on the fly; (2) a dynamic embedding (and unembedding) layer that computes embeddings for newly formed hypertokens at runtime; and (3) a variant of autoregressive language modeling that pretrains the model to handle hypertokenized, compressed text sequences as inputs and outputs. We show that an existing LLM can be uptrained for zip2zip in 10 GPU-hours via parameter-efficient finetuning. The resulting LLM performs test-time adaptation, learning to use hypertokens in unseen contexts and reducing input and output tokens by 15-40%.

Method: Uses adversarial interactions between LLM anonymizer and inference model to collect anonymized texts and inferred attributes, then distills anonymization and critique capabilities into SLMs through supervised fine-tuning and preference learning with self-refinement.

Result: 8B SLMs achieve privacy-utility trade-off comparable to GPT-4 anonymizer, and with self-refinement surpass GPT-4 in privacy protection on SynthPAI dataset.

Conclusion: SEAL’s adversarial distillation framework effectively trains SLMs as efficient anonymizers, providing strong privacy protection without external model dependencies.

Abstract: Large language models (LLMs) are increasingly used in sensitive domains, where their ability to infer personal data from seemingly benign text introduces emerging privacy risks. While recent LLM-based anonymization methods help mitigate such risks, they often rely on proprietary models (e.g., GPT-4), raising concerns about cost and the potential exposure of sensitive data to untrusted external systems. To address this, we introduce SElf-refining Anonymization with Language model (SEAL), a novel distillation framework for training small language models (SLMs) to perform effective anonymization without relying on external models at inference time. SEAL leverages adversarial interactions between an LLM anonymizer and an inference model to collect trajectories of anonymized texts and inferred attributes, which are then used to distill anonymization and critique capabilities into SLMs through supervised fine-tuning and preference learning. The resulting models learn both to anonymize text and to evaluate their outputs, enabling iterative improvement of anonymization quality via self-refinement. Experiments on SynthPAI, a dataset of synthetic personal profiles and text comments, demonstrate that SLMs trained with SEAL achieve substantial improvements in anonymization capabilities. Notably, 8B models attain a privacy-utility trade-off comparable to that of the GPT-4 anonymizer and, with self-refinement, even surpass it in terms of privacy protection. These results highlight the effectiveness of our adversarial distillation framework for training SLMs as efficient anonymizers.

Method: Two-stage framework: 1) Gradient-based comparisons with teacher model identify confounding tokens, 2) Pruning these tokens during distillation to align student’s attention with teacher’s focus on critical context tokens.

Result: Achieves absolute improvement in mathematical reasoning, code generation and multi-hop QA benchmarks; suppresses attention to confounding tokens; yields more interpretable and reliable reasoning model.

Conclusion: LeaF effectively mitigates attention distraction in LLMs through intervention-based inference, improving reasoning accuracy and generation quality while reducing inference overhead.

Abstract: Large language models (LLMs) have demonstrated significant improvements in contextual understanding. However, their ability to attend to truly critical information during long-context reasoning and generation still falls behind the pace. Specifically, our preliminary experiments reveal that certain distracting patterns can misdirect the model’s attention during inference, and removing these patterns substantially improves reasoning accuracy and generation quality. We attribute this phenomenon to spurious correlations in the training data, which obstruct the model’s capacity to infer authentic causal instruction-response relationships. This phenomenon may induce redundant reasoning processes, potentially resulting in significant inference overhead and, more critically, the generation of erroneous or suboptimal responses. To mitigate this, we introduce a two-stage framework called Learning to Focus (LeaF) leveraging intervention-based inference to disentangle confounding factors. In the first stage, LeaF employs gradient-based comparisons with an advanced teacher to automatically identify confounding tokens based on causal relationships in the training corpus. Then, in the second stage, it prunes these tokens during distillation to enact intervention, aligning the student’s attention with the teacher’s focus distribution on truly critical context tokens. Experimental results demonstrate that LeaF not only achieves an absolute improvement in various mathematical reasoning, code generation and multi-hop question answering benchmarks but also effectively suppresses attention to confounding tokens during inference, yielding a more interpretable and reliable reasoning model.

Method: The authors developed a novel probing technique that can decode numeric values from input embeddings with near-perfect accuracy. This method is specifically designed to work with the sinusoidal patterns that emerge in learned number embeddings across various open-source language models.

Result: The new probing method achieves near-perfect accuracy in decoding numeric values, proving that language models represent numbers with remarkable precision after pre-training. The precision of these embeddings explains a large portion of arithmetic errors in LMs.

Conclusion: Aligning the embeddings with the patterns discovered by the new probing method can mitigate arithmetic errors in language models. This demonstrates that the issue is not with the representation quality but with how these representations are utilized.

Abstract: Pretrained language models (LMs) are prone to arithmetic errors. Existing work showed limited success in probing numeric values from models' representations, indicating that these errors can be attributed to the inherent unreliability of distributionally learned embeddings in representing exact quantities. However, we observe that previous probing methods are inadequate for the emergent structure of learned number embeddings with sinusoidal patterns. In response, we propose a novel probing technique that decodes numeric values from input embeddings with near-perfect accuracy across a range of open-source LMs. This proves that after the sole pre-training, LMs represent numbers with remarkable precision. Finally, we find that the embeddings’ precision, judged by our probe’s accuracy, explains a large portion of LM’s errors in elementary arithmetic, and show that aligning the embeddings with the pattern our probes discover can mitigate these errors.

Method: Uses reinforcement learning with an LLM router that interleaves ’think’ actions (internal deliberation) and ‘route’ actions (dynamic model invocation), with rule-based rewards for format, outcome, and cost optimization.

Result: Outperforms strong baselines on seven general and multi-hop QA benchmarks, achieving superior performance with robust generalization and cost management.

Conclusion: Router-R1 enables effective multi-LLM routing through sequential decision-making, balancing performance and cost while generalizing well to unseen models.

Abstract: The rapid emergence of diverse large language models (LLMs) has spurred the development of LLM routers that assign user queries to the most suitable model. However, existing LLM routers typically perform a single-round, one-to-one mapping (\textit{i.e.}, assigning each query to a single model in isolation), which limits their capability to tackle complex tasks that demand the complementary strengths of multiple LLMs. In this paper, we present \textbf{Router-R1}, a reinforcement learning (RL)-based framework that formulates multi-LLM routing and aggregation as a sequential decision process. Router-R1 instantiates the router itself as a capable LLM, leveraging its reasoning ability to interleave “think” actions (internal deliberation) with “route” actions (dynamic model invocation), and integrates each response into its evolving context. To facilitate learning, we employ a lightweight rule-based reward comprising format rewards, final outcome rewards, and a novel cost reward for optimizing the balance between performance and cost, opening a pathway toward enhancing performance-cost trade-offs via RL. Router-R1 also conditions only on simple model descriptors such as pricing, latency, and example performance, enabling strong generalization to unseen model selection. Experiments on seven general and multi-hop QA benchmarks show that Router-R1 outperforms several strong baselines, achieving superior performance while maintaining robust generalization and cost management.

Method: Used Erotetic Theory of Reasoning (ETR) and PyETR to generate 383 reasoning problems, evaluated 38 models, analyzed correctness and fallacy patterns, and tested premise order effects.

Result: Higher capability models produce more ETR-predicted fallacies in their errors, while overall correctness doesn’t correlate with capability. Reversing premise order reduces fallacy production, mirroring human order effects.

Conclusion: Language models increasingly exhibit human-like fallacy patterns as they become more capable, and PyETR provides a valuable framework for analyzing reasoning errors beyond simple accuracy metrics.

Abstract: We study logical reasoning in language models by asking whether their errors follow established human fallacy patterns. Using the Erotetic Theory of Reasoning (ETR) and its open-source implementation, PyETR, we programmatically generate 383 formally specified reasoning problems and evaluate 38 models. For each response, we judge logical correctness and, when incorrect, whether it matches an ETR-predicted fallacy. Two results stand out: (i) as a capability proxy (Chatbot Arena Elo) increases, a larger share of a model’s incorrect answers are ETR-predicted fallacies $(\rho=0.360, p=0.0265)$, while overall correctness on this dataset shows no correlation with capability; (ii) reversing premise order significantly reduces fallacy production for many models, mirroring human order effects. Methodologically, PyETR provides an open-source pipeline for unbounded, synthetic, contamination-resistant reasoning tests linked to a cognitive theory, enabling analyses that focus on error composition rather than error rate.

Method: The authors introduce DiscoSG-DS dataset with 400 expert-annotated and 8,430 synthesized multi-sentence caption-graph pairs, and propose DiscoSG-Refiner - a lightweight parser that drafts a seed graph and iteratively refines it using a learned graph-editing model.

Result: Fine-tuning GPT-4o on DiscoSG-DS yields over 40% higher SPICE than sentence-merging baselines. DiscoSG-Refiner achieves 30% higher SPICE than baselines while being 86 times faster than GPT-4o, and consistently improves downstream VLM tasks including caption evaluation and hallucination detection.

Conclusion: DiscoSG-Refiner effectively bridges the gap between expensive large models and simpler open-source parsers, providing high-quality discourse-level scene graph parsing that generalizes from simple to dense graphs and improves downstream VLM applications.

Abstract: Vision-Language Models (VLMs) generate discourse-level, multi-sentence visual descriptions, challenging text scene graph parsers built for single-sentence caption-to-graph mapping. Current approaches typically merge sentence-level parsing outputs for discourse input, often missing phenomena like cross-sentence coreference, resulting in fragmented graphs and degraded downstream VLM task performance. We introduce a new task, Discourse-level text Scene Graph parsing (DiscoSG), and release DiscoSG-DS, a dataset of 400 expert-annotated and 8,430 synthesised multi-sentence caption-graph pairs. Each caption averages 9 sentences, and each graph contains at least 3 times more triples than those in existing datasets. Fine-tuning GPT-4o on DiscoSG-DS yields over 40% higher SPICE metric than the best sentence-merging baseline. However, its high inference cost and licensing restrict open-source use. Smaller fine-tuned open-source models (e.g., Flan-T5) perform well on simpler graphs yet degrade on denser, more complex graphs. To bridge this gap, we introduce DiscoSG-Refiner, a lightweight open-source parser that drafts a seed graph and iteratively refines it with a novel learned graph-editing model, achieving 30% higher SPICE than the baseline while delivering 86 times faster inference than GPT-4o. It generalises from simple to dense graphs, thereby consistently improving downstream VLM tasks, including discourse-level caption evaluation and hallucination detection, outperforming alternative open-source parsers. Code and data are available at https://github.com/ShaoqLin/DiscoSG .

Method: Theoretical proof showing transformers with fixed precision (except attention) are equivalent to C-RASP programs, then proving deeper C-RASP programs are more expressive. Also studied transformers with positional encodings and provided empirical validation.

Result: Proved that deeper transformers are more expressive than shallower transformers within the studied subclass, with similar results for transformers with positional encodings. Empirical evidence supports the theory.

Conclusion: Depth increases transformer expressivity, and the theory predicts the required depth for length generalization on sequential dependency tasks.

Abstract: It has been observed that transformers with greater depth (that is, more layers) have more capabilities, but can we establish formally which capabilities are gained? We answer this question with a theoretical proof followed by an empirical study. First, we consider transformers that round to fixed precision except inside attention. We show that this subclass of transformers is expressively equivalent to the programming language C-RASP and this equivalence preserves depth. Second, we prove that deeper C-RASP programs are more expressive than shallower C-RASP programs, implying that deeper transformers are more expressive than shallower transformers (within the subclass mentioned above). The same is also proven for transformers with positional encodings (like RoPE and ALiBi). These results are established by studying a temporal logic with counting operators equivalent to C-RASP. Finally, we provide empirical evidence that our theory predicts the depth required for transformers without positional encodings to length-generalize on a family of sequential dependency tasks.

Method: Used prompt-only interventions including deontological moral framing and scratchpad reasoning on LLaMA 3 8B model, and introduced a taxonomy distinguishing shallow vs deep deception.

Result: Demonstrated that alignment faking occurs even in small models and can be significantly reduced through prompting interventions, challenging scale-based assumptions about deceptive alignment.

Conclusion: Findings refine understanding of deception in language models and underscore the need for alignment evaluations across different model sizes and deployment settings.

Abstract: Current literature suggests that alignment faking (deceptive alignment) is an emergent property of large language models. We present the first empirical evidence that a small instruction-tuned model, specifically LLaMA 3 8B, can exhibit alignment faking. We further show that prompt-only interventions, including deontological moral framing and scratchpad reasoning, significantly reduce this behavior without modifying model internals. This challenges the assumption that prompt-based ethics are trivial and that deceptive alignment requires scale. We introduce a taxonomy distinguishing shallow deception, shaped by context and suppressible through prompting, from deep deception, which reflects persistent, goal-driven misalignment. Our findings refine the understanding of deception in language models and underscore the need for alignment evaluations across model sizes and deployment settings.

Method: Uses retrieval-augmented generation grounded in university resources, with a custom FAQ retriever that maps user questions to knowledge-base entries, improving over standard dense/hybrid retrieval strategies.

Result: The system is engineered for easy deployment in resource-constrained academic settings and has been technically evaluated with real-world deployment insights.

Conclusion: Marcel successfully provides an effective solution for handling admission inquiries through its lightweight, open-source architecture that delivers verifiable, contextually relevant information while reducing staff burden.

Abstract: We present Marcel, a lightweight and open-source conversational agent designed to support prospective students with admission-related inquiries. The system aims to provide fast and personalized responses, while reducing workload of university staff. We employ retrieval-augmented generation to ground answers in university resources and to provide users with verifiable, contextually relevant information. We introduce a Frequently Asked Question (FAQ) retriever that maps user questions to knowledge-base entries, which allows administrators to steer retrieval, and improves over standard dense/hybrid retrieval strategies. The system is engineered for easy deployment in resource-constrained academic settings. We detail the system architecture, provide a technical evaluation of its components, and report insights from a real-world deployment.

Method: Systematic investigation of interactions between fine-tuning objectives and model editing techniques, testing retention of edited vs intrinsic knowledge.

Result: Edited knowledge is substantially more susceptible to forgetting during fine-tuning than intrinsic pre-trained knowledge. Knowledge retention can be improved through paraphrasing augmentations or freezing layers associated with edited content.

Conclusion: Current model editing approaches have limitations in robustness under fine-tuning, and evaluating edit robustness during downstream fine-tuning is critical for practical deployment. The findings provide insights for developing more robust editing algorithms.

Abstract: Large language models (LLMs) store vast amounts of knowledge, which often requires updates to correct factual errors, incorporate newly acquired information, or adapt model behavior. Model editing methods have emerged as efficient solutions for such updates, offering localized and precise knowledge modification at significantly lower computational cost than continual training. In parallel, LLMs are frequently fine-tuned for a wide range of downstream tasks. However, the effect of fine-tuning on previously edited knowledge remains poorly understood. In this work, we systematically investigate how different fine-tuning objectives interact with various model editing techniques. Our findings show that edited knowledge is substantially more susceptible to forgetting during fine-tuning than intrinsic knowledge acquired through pre-training. This analysis highlights a key limitation of current editing approaches and suggests that evaluating edit robustness under downstream fine-tuning is critical for their practical deployment. We further find that knowledge retention can be significantly improved by either augmenting edit knowledge with paraphrases or by freezing layers associated with edited content in fine-tuning stage, offering insight for developing more robust editing algorithms.

Method: Proposed Magical with asymmetric LoRA architecture: shared matrix A for abstractive summarization, multiple isolated matrices B for diverse lay-style generation, Semantic Invariance Constraint to preserve semantic fidelity, and Recommendation-guided Switch to prompt switching between different B matrices.

Result: Experimental results on three real-world datasets show Magical consistently outperforms prompt-based methods, vanilla LoRA, and its recent variants while reducing trainable parameters by 31.66%.

Conclusion: Magical effectively addresses the limitations of standard LoRA in heterogeneous MLLG scenarios, achieving better performance with fewer parameters through its asymmetric architecture and semantic preservation mechanisms.

Abstract: Medical Lay Language Generation (MLLG) plays a vital role in improving the accessibility of complex scientific content for broader audiences. Recent literature to MLLG commonly employ parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) to fine-tuning large language models (LLMs) using paired expert-lay language datasets. However, LoRA struggles with the challenges posed by multi-source heterogeneous MLLG datasets. Specifically, through a series of exploratory experiments, we reveal that standard LoRA fail to meet the requirement for semantic fidelity and diverse lay-style generation in MLLG task. To address these limitations, we propose Magical, an asymmetric LoRA architecture tailored for MLLG under heterogeneous data scenarios. Magical employs a shared matrix $A$ for abstractive summarization, along with multiple isolated matrices $B$ for diverse lay-style generation. To preserve semantic fidelity during the lay language generation process, Magical introduces a Semantic Invariance Constraint to mitigate semantic subspace shifts on matrix $A$. Furthermore, to better adapt to diverse lay-style generation, Magical incorporates the Recommendation-guided Switch, an externally interface to prompt the LLM to switch between different matrices $B$. Experimental results on three real-world lay language generation datasets demonstrate that Magical consistently outperforms prompt-based methods, vanilla LoRA, and its recent variants, while also reducing trainable parameters by 31.66%. Our code is publicly available at https://github.com/tianlwang/Magical.git.

Method: GAICo offers a unified, extensible framework with comprehensive reference-based metrics for unstructured text, structured data, and multimedia (images, audio), featuring both high-level API for end-to-end analysis and direct metric access for granular control.

Result: GAICo demonstrated utility through a case study evaluating multi-modal AI Travel Assistant pipelines. The tool has been downloaded over 13K times since its PyPI release in June 2025, showing strong community adoption.

Conclusion: GAICo empowers researchers and developers to efficiently assess system performance, make evaluation reproducible, improve development velocity, and build more trustworthy AI systems, enabling faster and safer AI deployment.

Abstract: The rapid proliferation of Generative AI (GenAI) into diverse, high-stakes domains necessitates robust and reproducible evaluation methods. However, practitioners often resort to ad-hoc, non-standardized scripts, as common metrics are often unsuitable for specialized, structured outputs (e.g., automated plans, time-series) or holistic comparison across modalities (e.g., text, audio, and image). This fragmentation hinders comparability and slows AI system development. To address this challenge, we present GAICo (Generative AI Comparator): a deployed, open-source Python library that streamlines and standardizes GenAI output comparison. GAICo provides a unified, extensible framework supporting a comprehensive suite of reference-based metrics for unstructured text, specialized structured data formats, and multimedia (images, audio). Its architecture features a high-level API for rapid, end-to-end analysis, from multi-model comparison to visualization and reporting, alongside direct metric access for granular control. We demonstrate GAICo’s utility through a detailed case study evaluating and debugging complex, multi-modal AI Travel Assistant pipelines. GAICo empowers AI researchers and developers to efficiently assess system performance, make evaluation reproducible, improve development velocity, and ultimately build more trustworthy AI systems, aligning with the goal of moving faster and safer in AI deployment. Since its release on PyPI in Jun 2025, the tool has been downloaded over 13K times, across versions, by Aug 2025, demonstrating growing community interest.

Method: Proposes a generative NEC method that utilizes speech sound features to retrieve candidate entities, then uses these features and candidates to annotate entity errors in ASR transcripts and replace them with correct entities.

Result: Tested on open-source and self-constructed datasets, the method shows significant improvement in entity accuracy, particularly effective in scenarios with word form differences.

Conclusion: The proposed generative annotation NEC method effectively addresses the limitation of existing phonetic-based approaches and improves entity transcription accuracy in ASR systems.

Abstract: End-to-end automatic speech recognition systems often fail to transcribe domain-specific named entities, causing catastrophic failures in downstream tasks. Numerous fast and lightweight named entity correction (NEC) models have been proposed in recent years. These models, mainly leveraging phonetic-level edit distance algorithms, have shown impressive performances. However, when the forms of the wrongly-transcribed words(s) and the ground-truth entity are significantly different, these methods often fail to locate the wrongly transcribed words in hypothesis, thus limiting their usage. We propose a novel NEC method that utilizes speech sound features to retrieve candidate entities. With speech sound features and candidate entities, we inovatively design a generative method to annotate entity errors in ASR transcripts and replace the text with correct entities. This method is effective in scenarios of word form difference. We test our method using open-source and self-constructed test sets. The results demonstrate that our NEC method can bring significant improvement to entity accuracy. The self-constructed training data and test set is publicly available at github.com/L6-NLP/Generative-Annotation-NEC.

Method: Analyzed a large-scale contemporary conversational corpus using machine learning and neural language models to estimate information density, comparing contextual word embeddings with traditional subcategorization probability measures.

Result: Replicated the established relationship between information density and ’that’-mentioning. Found that contextual word embeddings account for additional variance in complementizer usage patterns compared to previous subcategorization-based measures, which captured substantial idiosyncratic lexical variation.

Conclusion: Contextual word embeddings provide superior estimates of information density for predicting complementizer usage, advancing our understanding of how speakers maintain uniform information transmission in language production.

Abstract: Speakers often have multiple ways to express the same meaning. The Uniform Information Density (UID) hypothesis suggests that speakers exploit this variability to maintain a consistent rate of information transmission during language production. Building on prior work linking UID to syntactic reduction, we revisit the finding that the optional complementizer $\textit{that}$ in English complement clauses is more likely to be omitted when the clause has low information density (i.e., more predictable). We advance this line of research by analyzing a large-scale, contemporary conversational corpus and using machine learning and neural language models to refine estimates of information density. Our results replicated the established relationship between information density and $\textit{that}$-mentioning. However, we found that previous measures of information density based on matrix verbs’ subcategorization probability capture substantial idiosyncratic lexical variation. By contrast, estimates derived from contextual word embeddings account for additional variance in patterns of complementizer usage.

Method: Evaluated 7 state-of-the-art LVLMs as overhearers on a corpus of human conversations during collaborative object-matching tasks across multiple rounds.

Result: Current LVLMs perform poorly on this task and show no consistent improvement when exposed to more conversations from the same participants.

Conclusion: Understanding spontaneous referring expressions in conversational contexts remains a significant challenge for current LVLMs.

Abstract: During spontaneous conversations, speakers collaborate on novel referring expressions, which they can then re-use in subsequent conversations. Understanding such referring expressions is an important ability for an embodied agent, so that it can carry out tasks in the real world. This requires integrating and understanding language, vision, and conversational interaction. We study the capabilities of seven state-of-the-art Large Vision Language Models (LVLMs) as overhearers to a corpus of spontaneous conversations between pairs of human discourse participants engaged in a collaborative object-matching task. We find that such a task remains challenging for current LVLMs and they all fail to show a consistent performance improvement as they overhear more conversations from the same discourse participants repeating the same task for multiple rounds. We release our corpus and code for reproducibility and to facilitate future research.

Method: Introduces Convolutional decoding (Conv) - a normalization-based method that narrows the decoding window without hard segmentation, and Rejecting Rule-based Fine-Tuning (R2FT) - a post-hoc training scheme to better align tokens at positions far from context.

Result: Achieves state-of-the-art results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines with significantly lower step size than previous works, demonstrating both speed and quality improvements.

Conclusion: The proposed methods effectively overcome the long decoding-window problem in diffusion language models while maintaining their speed advantages, offering a promising alternative to autoregressive models.

Abstract: Autoregressive (AR) language models generate text one token at a time, which limits their inference speed. Diffusion-based language models offer a promising alternative, as they can decode multiple tokens in parallel. However, we identify a key bottleneck in current diffusion LMs: the long decoding-window problem, where tokens generated far from the input context often become irrelevant or repetitive. Previous solutions like semi-autoregressive address this issue by splitting windows into blocks (sacrificing bidirectionality), but we find that this also leads to time-interval expansion problem, sacrificing the speed. Therefore, semi-AR eliminates the main advantages of diffusion models. To overcome this, we propose Convolutional decoding (Conv), a normalization-based method that narrows the decoding window without hard segmentation, leading to better fluency and flexibility. Additionally, we introduce Rejecting Rule-based Fine-Tuning (R2FT), a post-hoc training scheme that better aligns tokens at positions far from context. Our methods achieve state-of-the-art results on open-ended generation benchmarks (e.g., AlpacaEval) among diffusion LM baselines, with significantly lower step size than previous works, demonstrating both speed and quality improvements.

Method: Reconceptualize context quality assessment as inference-time data valuation using CI value, which integrates query-aware relevance, list-aware uniqueness, and generator-aware alignment. Develop a parameterized surrogate model with hierarchical architecture for CI value prediction during inference.

Result: Extensive experiments across 8 NLP tasks and multiple LLMs show the method significantly outperforms state-of-the-art baselines, effectively filtering poor-quality contexts while preserving critical information.

Conclusion: The CI value-based context selection method provides a comprehensive solution for improving RAG performance by holistically assessing context quality and eliminating complex hyperparameter tuning requirements.

Abstract: Retrieval-Augmented Generation (RAG) addresses large language model (LLM) hallucinations by grounding responses in external knowledge, but its effectiveness is compromised by poor-quality retrieved contexts containing irrelevant or noisy information. While existing approaches attempt to improve performance through context selection based on predefined context quality assessment metrics, they show limited gains over standard RAG. We attribute this limitation to their failure in holistically utilizing available information (query, context list, and generator) for comprehensive quality assessment. Inspired by recent advances in data selection, we reconceptualize context quality assessment as an inference-time data valuation problem and introduce the Contextual Influence Value (CI value). This novel metric quantifies context quality by measuring the performance degradation when removing each context from the list, effectively integrating query-aware relevance, list-aware uniqueness, and generator-aware alignment. Moreover, CI value eliminates complex selection hyperparameter tuning by simply retaining contexts with positive CI values. To address practical challenges of label dependency and computational overhead, we develop a parameterized surrogate model for CI value prediction during inference. The model employs a hierarchical architecture that captures both local query-context relevance and global inter-context interactions, trained through oracle CI value supervision and end-to-end generator feedback. Extensive experiments across 8 NLP tasks and multiple LLMs demonstrate that our context selection method significantly outperforms state-of-the-art baselines, effectively filtering poor-quality contexts while preserving critical information. Code is available at https://github.com/SJTU-DMTai/RAG-CSM.

Method: Proposed CARB benchmark covering 10 cultures across 4 domains, and Think-as-Locals approach using reinforcement learning from verifiable rewards (RLVR) to elicit deeper cultural reasoning.

Result: Current RMs show deficiencies in cultural awareness modeling, with scoring relying on surface-level features rather than cultural nuance. CARB performance correlates with downstream multilingual cultural alignment.

Conclusion: Think-as-Locals with RLVR effectively mitigates spurious feature interference and advances culture-aware reward modeling by ensuring accurate preference judgments and high-quality evaluation criteria.

Abstract: Reward models (RMs) are crucial for aligning large language models (LLMs) with diverse cultures. Consequently, evaluating their cultural awareness is essential for further advancing global alignment of LLMs. However, existing RM evaluations fall short in assessing cultural awareness due to the scarcity of culturally relevant evaluation datasets. To fill this gap, we propose Cultural Awareness Reward modeling Benchmark (CARB), covering 10 distinct cultures across 4 cultural domains. Our extensive evaluation of state-of-the-art RMs reveals their deficiencies in modeling cultural awareness and demonstrates a positive correlation between performance on CARB and downstream multilingual cultural alignment tasks. Further analysis identifies the spurious correlations within culture-aware reward modeling, wherein RM’s scoring relies predominantly on surface-level features rather than authentic cultural nuance understanding. To address these, we propose Think-as-Locals to elicit deeper culturally grounded reasoning from generative RMs via reinforcement learning from verifiable rewards (RLVR) and employ well-designed rewards to ensure accurate preference judgments and high-quality structured evaluation criteria generation. Experimental results validate its efficacy in mitigating spurious features interference and advancing culture-aware reward modeling.

Method: Pretrains language models to first generate intermediate latent thoughts (last hidden states) which are then used as input to predict subsequent tokens, adding computational refinement steps during pretraining.

Result: PonderLM-2-Pythia-1.4B outperforms vanilla Pythia-2.8B on language modeling and downstream tasks despite having half the parameters. Performance consistently improves with more latent thoughts per token.

Conclusion: Scaling computational steps during pretraining through latent thought generation significantly enhances language model performance, achieving better results than simply scaling model parameters.

Abstract: The remarkable success of Chain-of-Thought (CoT), which enhances performance by scaling generation steps at test-time, inspires us to ask: can we leverage a similar scaling of computational steps during pretraining to improve the generation of each individual token? To address this, we propose a novel pre-training methodology: Pretraining Language Models with Latent Thoughts (PonderLM-2). Our approach pretrains a language model (LM) to first generate an intermediate latent thought-the last hidden state of the current position-which is then used as input to predict the actual subsequent token. This additional computational step enables the LM to refine its prediction within unconstrained continuous space. Our experiments demonstrate that, at an identical inference cost, a LM that generates one additional latent thought per token outperforms a standard model with double the parameters. For instance, our PonderLM-2-Pythia-1.4B, pretrained on 300B tokens from the Pile, significantly surpasses the vanilla Pythia-2.8B trained on the same data on both language modeling and a range of general downstream tasks. Furthermore, increasing the number of latent thoughts generated before each actual token-forming a chain analogous to CoT-consistently improves the model’s performance.

Method: Three-phase methodology: systematic error profiling across hierarchical layers (knowledge gaps, reasoning limitations, complexity constraints), targeted intervention design matched to dominant error sources, and rigorous validation with statistical testing.

Result: Average accuracy gains of 11.2 percentage points (p < .001) across health communication, political communication, and legal tasks using five diverse LLMs, with consistent improvements (range: +6.8 to +14.6pp) and reduced systematic misclassification.

Conclusion: HEC provides a transparent, measurement-first blueprint for diagnosing error profiles, selecting targeted interventions, and reporting reliability/validity evidence, applicable to automated coding across communication research and social sciences, though with diminished returns in very high-baseline tasks.

Abstract: Automated content analysis increasingly supports communication research, yet scaling manual coding into computational pipelines raises concerns about measurement reliability and validity. We introduce a Hierarchical Error Correction (HEC) framework that treats model failures as layered measurement errors (knowledge gaps, reasoning limitations, and complexity constraints) and targets the layers that most affect inference. The framework implements a three-phase methodology: systematic error profiling across hierarchical layers, targeted intervention design matched to dominant error sources, and rigorous validation with statistical testing. Evaluating HEC across health communication (medical specialty classification) and political communication (bias detection), and legal tasks, we validate the approach with five diverse large language models. Results show average accuracy gains of 11.2 percentage points (p < .001, McNemar’s test) and stable conclusions via reduced systematic misclassification. Cross-model validation demonstrates consistent improvements (range: +6.8 to +14.6pp), with effectiveness concentrated in moderate-to-high baseline tasks (50-85% accuracy). A boundary study reveals diminished returns in very high-baseline (>85%) or precision-matching tasks, establishing applicability limits. We map layered errors to threats to construct and criterion validity and provide a transparent, measurement-first blueprint for diagnosing error profiles, selecting targeted interventions, and reporting reliability/validity evidence alongside accuracy. This applies to automated coding across communication research and the broader social sciences.

Method: Adaptively learns a witness function from training data to enhance logits-based detectors, with statistical guarantees on performance metrics.

Result: Extensive studies show AdaDetectGPT nearly uniformly improves state-of-the-art methods across various datasets and LLMs, with improvements up to 37%.

Conclusion: AdaDetectGPT provides a more effective approach for LLM-generated text detection by adaptively learning from data rather than relying solely on log probabilities.

Abstract: We study the problem of determining whether a piece of text has been authored by a human or by a large language model (LLM). Existing state of the art logits-based detectors make use of statistics derived from the log-probability of the observed text evaluated using the distribution function of a given source LLM. However, relying solely on log probabilities can be sub-optimal. In response, we introduce AdaDetectGPT – a novel classifier that adaptively learns a witness function from training data to enhance the performance of logits-based detectors. We provide statistical guarantees on its true positive rate, false positive rate, true negative rate and false negative rate. Extensive numerical studies show AdaDetectGPT nearly uniformly improves the state-of-the-art method in various combination of datasets and LLMs, and the improvement can reach up to 37%. A python implementation of our method is available at https://github.com/Mamba413/AdaDetectGPT.

Method: The method introduces a task-agnostic token-level uncertainty measure, decomposes it to isolate epistemic uncertainty, and approximates the true distribution using an idealized model. For contextual QA, it extracts three semantic features (context-reliance, context comprehension, and honesty) using a top-down interpretability approach with minimal labeled samples.

Result: Experiments on multiple QA benchmarks in both in-distribution and out-of-distribution settings show the method substantially outperforms state-of-the-art unsupervised and supervised UQ methods, achieving up to 13-point PRR improvement with negligible inference overhead.

Conclusion: The proposed theoretically grounded approach effectively quantifies epistemic uncertainty in contextual QA by identifying semantic feature gaps, demonstrating superior performance over existing methods across various settings.

Abstract: Uncertainty Quantification (UQ) research has primarily focused on closed-book factual question answering (QA), while contextual QA remains unexplored, despite its importance in real-world applications. In this work, we focus on UQ for the contextual QA task and propose a theoretically grounded approach to quantify epistemic uncertainty. We begin by introducing a task-agnostic, token-level uncertainty measure defined as the cross-entropy between the predictive distribution of the given model and the unknown true distribution. By decomposing this measure, we isolate the epistemic component and approximate the true distribution by a perfectly prompted, idealized model. We then derive an upper bound for epistemic uncertainty and show that it can be interpreted as semantic feature gaps in the given model’s hidden representations relative to the ideal model. We further apply this generic framework to the contextual QA task and hypothesize that three features approximate this gap: context-reliance (using the provided context rather than parametric knowledge), context comprehension (extracting relevant information from context), and honesty (avoiding intentional lies). Using a top-down interpretability approach, we extract these features by using only a small number of labeled samples and ensemble them to form a robust uncertainty score. Experiments on multiple QA benchmarks in both in-distribution and out-of-distribution settings show that our method substantially outperforms state-of-the-art unsupervised (sampling-free and sampling-based) and supervised UQ methods, achieving up to a 13-point PRR improvement while incurring a negligible inference overhead.

Method: Created RECODE-H benchmark with 102 tasks, structured instructions, unit tests, and a five-level feedback hierarchy. Developed ReCodeAgent framework that integrates feedback into iterative code generation.

Result: Experiments with leading LLMs (GPT-5, Claude-Sonnet-4, DeepSeek-V3.1, Gemini 2.5) show substantial performance gains with richer feedback, though challenges remain in generating complex research code.

Conclusion: RECODE-H establishes a foundation for developing adaptive, feedback-driven LLM agents in scientific research implementation.

Abstract: Large language models (LLMs) show the promise in supporting scientific research implementation, yet their ability to generate correct and executable code remains limited. Existing works largely adopt one-shot settings, ignoring the iterative and feedback-driven nature of realistic workflows of scientific research development. To address this gap, we present RECODE-H, a benchmark of 102 tasks from research papers and repositories that evaluates LLM agents through multi-turn interactions with LLM-simulated human feedback. It includes structured instructions,unit tests, and a five-level feedback hierarchy to reflect realistic researcher-agent collaboration. We further present ReCodeAgent, a framework that integrates feedback into iterative code generation. Experiments with leading LLMs, including GPT-5, Claude-Sonnet-4, DeepSeek-V3.1, and Gemini 2.5, show substantial performance gains with richer feedback, while also highlighting ongoing challenges in the generation of complex research code. RECODE-H establishes a foundation for developing adaptive, feedback-driven LLM agents in scientific research implementation

Method: Decomposes questions into ordered sub-questions, retrieves relevant triples from graph, dynamically expands graph with new triples from documents, and aggregates triples into graph memory.

Result: Achieves consistent and significant improvements on three multi-hop QA benchmarks, especially in Exact Match scores.

Conclusion: SubQRAG effectively enhances reasoning depth in Graph RAG through sub-question decomposition and dynamic graph expansion.

Abstract: Graph Retrieval-Augmented Generation (Graph RAG) effectively builds a knowledge graph (KG) to connect disparate facts across a large document corpus. However, this broad-view approach often lacks the deep structured reasoning needed for complex multi-hop question answering (QA), leading to incomplete evidence and error accumulation. To address these limitations, we propose SubQRAG, a sub-question-driven framework that enhances reasoning depth. SubQRAG decomposes a complex question into an ordered chain of verifiable sub-questions. For each sub-question, it retrieves relevant triples from the graph. When the existing graph is insufficient, the system dynamically expands it by extracting new triples from source documents in real time. All triples used in the reasoning process are aggregated into a “graph memory,” forming a structured and traceable evidence path for final answer generation. Experiments on three multi-hop QA benchmarks demonstrate that SubQRAG achieves consistent and significant improvements, especially in Exact Match scores.

Method: Extracts building blocks of cognition from prior examples to create abstracted schemas - lightweight structured templates of key inferential steps and their relationships - which augment the model’s reasoning process for novel questions.

Result: SA-ICL consistently boosts performance up to 36.19% on chemistry and physics questions from GPQA dataset, reduces reliance on demonstration count, and enhances interpretability. Most LLMs lack implicit schema formation but benefit from explicit schema scaffolding.

Conclusion: SA-ICL bridges disparate ICL strategies and paves a new path for enhancing human-like reasoning in LLMs by providing explicit schema-based scaffolding that mimics cognitive processes.

Abstract: In-Context Learning (ICL) enables transformer-based language models to adapt to new tasks by conditioning on demonstration examples. However, traditional example-driven in-context learning lacks explicit modules for knowledge retrieval and transfer at the abstraction level. Inspired by cognitive science, specifically schema theory, which holds that humans interpret new information by activating pre-existing mental frameworks (schemas) to structure understanding, we introduce SCHEMA ACTIVATED IN CONTEXT LEARNING (SA-ICL). This framework extracts the representation of the building blocks of cognition for the reasoning process instilled from prior examples, creating an abstracted schema, a lightweight, structured template of key inferential steps and their relationships, which is then used to augment a model’s reasoning process when presented with a novel question. We demonstrate that a broad range of large language models (LLMs) lack the capacity to form and utilize internal schema-based learning representations implicitly, but instead benefit significantly from explicit schema-based scaffolding. Across chemistry and physics questions from the GPQA dataset, our experiments show that SA-ICL consistently boosts performance, up to 36.19 percent, when the single demonstration example is of high quality, which simultaneously reduces reliance on the number of demonstrations and enhances interpretability. SCHEMA ACTIVATED IN CONTEXT LEARNING not only bridges disparate ICL strategies ranging from pattern priming to Chain-of-Thought prompting, but also paves a new path for enhancing human-like reasoning in LLMs.

Method: Propose Chain of Conceptual Thoughts (CoCT) where LLMs first generate concept tags (emotions, strategies, topics) then complete detailed content following these concepts.

Result: CoCT outperforms baselines like self-refine, ECoT, SoT and RAG in daily and emotional support conversations across in-domain and out-of-domain concept settings.

Conclusion: CoCT provides a potential solution for LLM prompting paradigm that can be applied to a wider scope of tasks beyond traditional reasoning domains.

Abstract: Chain-of-Thought (CoT) is widely applied to enhance the LLM capability in math, coding and reasoning tasks. However, its performance is limited for open-domain tasks, when there are no clearly defined reasoning steps or logical transitions. To mitigate such challenges, we propose a new prompt-based paradigm called Chain of Conceptual Thoughts (CoCT), which suggests the LLM first to produce the tag of concepts, then complete the detailed content following the concept. To encourage this hierarchical way of thinking, we implement the concepts with emotions, strategies and topics. We experiment with this paradigm in daily and emotional support conversations, covering tasks with both in-domain and out-of-domain concept settings. Automatic, human, and LLM-based evaluations reveal that CoCT surpasses several prompt-based baselines such as self-refine, ECoT, SoT and RAG, suggesting a potential solution of LLM prompting paradigm for a wider scope of tasks.

Method: DePass decomposes hidden states into customized additive components and propagates them through the model with attention scores and MLP activations fixed, requiring no auxiliary training.

Result: Validated across token-level, model component-level, and subspace-level attribution tasks, demonstrating effectiveness and fidelity in attributing information flow between arbitrary Transformer components.

Conclusion: DePass serves as a foundational tool for broader interpretability applications, enabling faithful attribution of information flow in Transformer models.

Abstract: Attributing the behavior of Transformer models to internal computations is a central challenge in mechanistic interpretability. We introduce DePass, a unified framework for feature attribution based on a single decomposed forward pass. DePass decomposes hidden states into customized additive components, then propagates them with attention scores and MLP’s activations fixed. It achieves faithful, fine-grained attribution without requiring auxiliary training. We validate DePass across token-level, model component-level, and subspace-level attribution tasks, demonstrating its effectiveness and fidelity. Our experiments highlight its potential to attribute information flow between arbitrary components of a Transformer model. We hope DePass serves as a foundational tool for broader applications in interpretability.

Method: Developed GEMs family with architectural diversity (RoBERTa, Longformer, ELECTRA, ConvBERT, ModernBERT) and enhanced data curation including preprocessing, deduplication, and targeted repetition of high-quality legal sub-corpora. Also created first bilingual Greek-English models for cross-lingual legal applications.

Result: GEM-RoBERTa and GEM-ConvBERT achieved statistically significant performance improvements over state-of-the-art models with accuracy gains up to 3.6%, confirmed by Friedman Aligned-Ranks and Finner post-hoc tests across multiple evaluation metrics.

Conclusion: The proposed GEMs family successfully addresses Greek language modeling limitations through architectural diversity and sophisticated data curation, establishing new state-of-the-art performance for Modern Greek NLP.

Abstract: The advancement of natural language processing for morphologically rich and moderately-resourced languages like Modern Greek has been hindered by architectural stagnation, data scarcity, and limited context processing capabilities, particularly in specialized domains such as law. In this work, we propose the Greek Embedding Models (GEMs), a new family of transformer-based language models, specifically developed to address these limitations through architectural diversity and enhanced data curation. The proposed family of models are trained on several large-scale, meticulously curated corpora, encompassing both comprehensive general-domain datasets and specialized legal collections, addressing the persistent data scarcity that has impeded Greek language modeling advancement. The proposed quality-based corpus curation methodology incorporates extensive preprocessing pipelines, sophisticated deduplication strategies and targeted repetition of high-quality legal sub-corpora to enhance domain adaptation. The GEMs family comprises both established architectures (RoBERTa and Longformer) and advanced models not previously applied to Greek (ELECTRA, ConvBERT, and ModernBERT), providing comprehensive coverage of modern transformer designs. Additionally, we introduce the first bilingual Greek-English embedding models tailored for cross-lingual legal applications. Comprehensive evaluation across three core natural language understanding benchmarks demonstrates that the proposed GEM-RoBERTa and GEM-ConvBERT achieve statistically significant performance improvements over established state-of-the-art models, with accuracy gains of up to 3.6% while conducted statistical analysis using Friedman Aligned-Ranks and Finner post-hoc tests confirms the superiority of our approach across multiple evaluation metrics.

Method: A multi-agent framework with three specialized roles (planning, coding, answering) that uses code execution for table manipulation. Includes self-improvement training using Monte Carlo Tree Search (MCTS) rollouts to generate pseudo-gold trajectories and optimize agents with reinforcement learning (RL).

Result: Achieved 62.13% on TableBench, surpassing OpenAI-o4-mini-high. Shows substantial gains in table understanding performance.

Conclusion: The framework demonstrates the promise of combining structured multi-agent workflows with reinforcement learning to advance table understanding, effectively integrating robust reasoning with reliable table processing.

Abstract: Understanding and reasoning over tables is a critical capability for many real-world applications. Large language models (LLMs) have shown promise on this task, but current approaches remain limited. Fine-tuning based methods strengthen language reasoning; yet they are prone to arithmetic errors and hallucination. In contrast, tool-based methods enable precise table manipulation but rely on rigid schemas and lack semantic understanding. These complementary drawbacks highlight the need for approaches that integrate robust reasoning with reliable table processing. In this work, we propose Mixture-of-Minds, a multi-agent framework that decomposes table reasoning into three specialized roles: planning, coding, and answering. This design enables each agent to focus on a specific aspect of the task while leveraging code execution for precise table manipulation. Building on this workflow, we introduce a self-improvement training framework that employs Monte Carlo Tree Search (MCTS) rollouts to generate pseudo-gold trajectories and optimize agents with reinforcement learning (RL). Extensive experiments show that Mixture-of-Minds delivers substantial gains, reaching 62.13% on TableBench and surpassing OpenAI-o4-mini-high. These results demonstrate the promise of combining structured multi-agent workflows with RL to advance table understanding.

Method: Robust Preference Selection (RPS) - a post-hoc method using directional neighborhood consensus. It samples multiple responses from related preference neighborhoods and selects the best aligned response.

Result: RPS consistently improves robustness across three alignment paradigms (DPA, DPO, SFT), achieving win rates up to 69% on challenging underrepresented preferences without retraining.

Conclusion: RPS provides a practical, theoretically-grounded solution for enhancing preference-aligned model reliability through training-free neighborhood sampling.

Abstract: Aligning large language models with human preferences is critical for creating reliable and controllable AI systems. A human preference can be visualized as a high-dimensional vector where different directions represent trade-offs between desired attributes (e.g., helpfulness vs. verbosity). Yet, because the training data often reflects dominant, average preferences, LLMs tend to perform well on common requests but fall short in specific, individual needs. This mismatch creates a preference coverage gap. Existing methods often address this through costly retraining, which may not be generalized to the full spectrum of diverse preferences. This brittleness means that when a user’s request reflects a nuanced preference deviating from the training data’s central tendency, model performance can degrade unpredictably. To address this challenge, we introduce Robust Preference Selection (RPS), a post-hoc, training-free method by leveraging directional neighborhood consensus. Instead of forcing a model to generate a response from a single, highly specific preference, RPS samples multiple responses from a local neighborhood of related preferences to create a superior candidate pool. It then selects the response that best aligns with the user’s original intent. We provide a theoretical framework showing our neighborhood generation strategy is provably superior to a strong baseline that also samples multiple candidates. Comprehensive experiments across three distinct alignment paradigms (DPA, DPO, and SFT) demonstrate that RPS consistently improves robustness against this baseline, achieving win rates of up to 69% on challenging preferences from under-represented regions of the space without any model retraining. Our work presents a practical, theoretically-grounded solution for enhancing the reliability of preference-aligned models.

Method: Interleave token mixers with intermediate complexity, including sparse attention with token eviction and query-aware native sparse attention. Propose learnable token eviction with sliding-window attention and lightweight CNN to retain critical KV-pairs.

Result: Empirical evaluations on retrieval-intensive benchmarks show effectiveness while maintaining linear attention’s constant time and space complexity.

Conclusion: Hybrid models with sparse attention mechanisms effectively address forgetfulness in linear-attention models for retrieval tasks.

Abstract: Linear-attention models that compress the entire input sequence into a fixed-size recurrent state offer an efficient alternative to Transformers, but their finite memory induces forgetfulness that harms retrieval-intensive tasks. To mitigate the issue, we explore a series of hybrid models that restore direct access to past tokens. We interleave token mixers with intermediate time and space complexity between linear and full attention, including sparse attention with token eviction, and the query-aware native sparse attention. Particularly, we propose a novel learnable token eviction approach. Combined with sliding-window attention, an end-to-end trainable lightweight CNN aggregates information from both past and future adjacent tokens to adaptively retain a limited set of critical KV-pairs per head, maintaining linear attention’s constant time and space complexity. Efficient Triton kernels for the sparse attention mechanisms are provided. Empirical evaluations on retrieval-intensive benchmarks support the effectiveness of our approaches.

Method: Route confounding factors through auxiliary modules (ControlNet or LoRA) during adapter training to eliminate incentive for the adapter to internalize them, then remove auxiliary modules during inference.

Result: Improves generation quality, diversity, and prompt adherence in facial and full-body identity injection tasks.

Conclusion: When seeking disentangled representations in large models, establishing shortcuts for what should NOT be learned is an effective design principle.

Abstract: Adapter-based training has emerged as a key mechanism for extending the capabilities of powerful foundation image generators, enabling personalized and stylized text-to-image synthesis. These adapters are typically trained to capture a specific target attribute, such as subject identity, using single-image reconstruction objectives. However, because the input image inevitably contains a mixture of visual factors, adapters are prone to entangle the target attribute with incidental ones, such as pose, expression, and lighting. This spurious correlation problem limits generalization and obstructs the model’s ability to adhere to the input text prompt. In this work, we uncover a simple yet effective solution: provide the very shortcuts we wish to eliminate during adapter training. In Shortcut-Rerouted Adapter Training, confounding factors are routed through auxiliary modules, such as ControlNet or LoRA, eliminating the incentive for the adapter to internalize them. The auxiliary modules are then removed during inference. When applied to tasks like facial and full-body identity injection, our approach improves generation quality, diversity, and prompt adherence. These results point to a general design principle in the era of large models: when seeking disentangled representations, the most effective path may be to establish shortcuts for what should NOT be learned.

Method: Uses Video-As-Prompt approach with reference videos as semantic prompts, guided by a frozen Video Diffusion Transformer via Mixture-of-Transformers expert and temporally biased position embedding for robust context retrieval.

Result: Achieves 38.7% user preference rate, rivaling leading condition-specific commercial models, with strong zero-shot generalization across various downstream applications.

Conclusion: VAP represents a significant advance toward general-purpose, controllable video generation with unified semantic control and strong generalization capabilities.

Abstract: Unified, generalizable semantic control in video generation remains a critical open challenge. Existing methods either introduce artifacts by enforcing inappropriate pixel-wise priors from structure-based controls, or rely on non-generalizable, condition-specific finetuning or task-specific architectures. We introduce Video-As-Prompt (VAP), a new paradigm that reframes this problem as in-context generation. VAP leverages a reference video as a direct semantic prompt, guiding a frozen Video Diffusion Transformer (DiT) via a plug-and-play Mixture-of-Transformers (MoT) expert. This architecture prevents catastrophic forgetting and is guided by a temporally biased position embedding that eliminates spurious mapping priors for robust context retrieval. To power this approach and catalyze future research, we built VAP-Data, the largest dataset for semantic-controlled video generation with over 100K paired videos across 100 semantic conditions. As a single unified model, VAP sets a new state-of-the-art for open-source methods, achieving a 38.7% user preference rate that rivals leading condition-specific commercial models. VAP’s strong zero-shot generalization and support for various downstream applications mark a significant advance toward general-purpose, controllable video generation.

Method: Proposes FM-BFF-Net that integrates convolutional and transformer components, uses focal modulation attention mechanism to refine context awareness, and introduces bidirectional feature fusion module for efficient interaction between encoder and decoder representations across scales.

Result: Extensive experiments on eight public datasets (polyp detection, skin lesion segmentation, ultrasound imaging) show FM-BFF-Net consistently outperforms state-of-the-art methods in Jaccard index and Dice coefficient, demonstrating enhanced boundary precision and robustness to lesion size, shape, and contrast variations.

Conclusion: FM-BFF-Net effectively addresses limitations of CNNs by combining transformer-based architecture with novel attention and fusion mechanisms, proving to be an effective and adaptable solution for diverse medical imaging segmentation tasks.

Abstract: Medical image segmentation is essential for clinical applications such as disease diagnosis, treatment planning, and disease development monitoring because it provides precise morphological and spatial information on anatomical structures that directly influence treatment decisions. Convolutional neural networks significantly impact image segmentation; however, since convolution operations are local, capturing global contextual information and long-range dependencies is still challenging. Their capacity to precisely segment structures with complicated borders and a variety of sizes is impacted by this restriction. Since transformers use self-attention methods to capture global context and long-range dependencies efficiently, integrating transformer-based architecture with CNNs is a feasible approach to overcoming these challenges. To address these challenges, we propose the Focal Modulation and Bidirectional Feature Fusion Network for Medical Image Segmentation, referred to as FM-BFF-Net in the remainder of this paper. The network combines convolutional and transformer components, employs a focal modulation attention mechanism to refine context awareness, and introduces a bidirectional feature fusion module that enables efficient interaction between encoder and decoder representations across scales. Through this design, FM-BFF-Net enhances boundary precision and robustness to variations in lesion size, shape, and contrast. Extensive experiments on eight publicly available datasets, including polyp detection, skin lesion segmentation, and ultrasound imaging, show that FM-BFF-Net consistently surpasses recent state-of-the-art methods in Jaccard index and Dice coefficient, confirming its effectiveness and adaptability for diverse medical imaging scenarios.

Method: Uses flow matching formulation with iterative refinement, window-dependent prior for consistency, and coordinate-specific variance schedule. During inference, employs best-first search on generated samples guided by model confidence.

Result: Achieves state-of-the-art tracking accuracy on occluded points while maintaining competitive performance on visible points across PointOdyssey, Dynamic Replica, and TAP-Vid benchmarks.

Conclusion: GenPT successfully models multi-modal trajectories and demonstrates superior performance in handling occlusions through its generative approach.

Abstract: Tracking a point through a video can be a challenging task due to uncertainty arising from visual obfuscations, such as appearance changes and occlusions. Although current state-of-the-art discriminative models excel in regressing long-term point trajectory estimates – even through occlusions – they are limited to regressing to a mean (or mode) in the presence of uncertainty, and fail to capture multi-modality. To overcome this limitation, we introduce Generative Point Tracker (GenPT), a generative framework for modelling multi-modal trajectories. GenPT is trained with a novel flow matching formulation that combines the iterative refinement of discriminative trackers, a window-dependent prior for cross-window consistency, and a variance schedule tuned specifically for point coordinates. We show how our model’s generative capabilities can be leveraged to improve point trajectory estimates by utilizing a best-first search strategy on generated samples during inference, guided by the model’s own confidence of its predictions. Empirically, we evaluate GenPT against the current state of the art on the standard PointOdyssey, Dynamic Replica, and TAP-Vid benchmarks. Further, we introduce a TAP-Vid variant with additional occlusions to assess occluded point tracking performance and highlight our model’s ability to capture multi-modality. GenPT is capable of capturing the multi-modality in point trajectories, which translates to state-of-the-art tracking accuracy on occluded points, while maintaining competitive tracking accuracy on visible points compared to extant discriminative point trackers.

Method: The authors introduce generalized median pooling and extend it by learning data-dependent pooling coefficients that adaptively weigh contributions from differently oriented pixel patches.

Result: Experimental results on various restoration benchmarks show that the anisotropic pooling strategy yields both perceptually and numerically superior results compared to existing LUT-realizable CNN methods.

Conclusion: Anisotropic pooling methods significantly improve the performance of LUT-based CNN restoration by better handling anisotropic signal structures through adaptive weighting of orientation contributions.

Abstract: Table look-up realization of image restoration CNNs has the potential of achieving competitive image quality while being much faster and resource frugal than the straightforward CNN implementation. The main technical challenge facing the LUT-based CNN algorithm designers is to manage the table size without overly restricting the receptive field. The prevailing strategy is to reuse the table for small pixel patches of different orientations (apparently assuming a degree of isotropy) and then fuse the look-up results. The fusion is currently done by average pooling, which we find being ill suited to anisotropic signal structures. To alleviate the problem, we investigate and discuss anisotropic pooling methods to replace naive averaging for improving the performance of the current LUT-realizable CNN restoration methods. First, we introduce the method of generalized median pooling which leads to measurable gains over average pooling. We then extend this idea by learning data-dependent pooling coefficients for each orientation, so that they can adaptively weigh the contributions of differently oriented pixel patches. Experimental results on various restoration benchmarks show that our anisotropic pooling strategy yields both perceptually and numerically superior results compared to existing LUT-realizable CNN methods.

Method: Created 3DReasonKnee dataset with 494k quintuples from 7,970 3D knee MRI volumes, each containing: 3D MRI volume, diagnostic question, 3D bounding box, clinician reasoning steps, and structured severity assessments. Involved 450+ hours of expert clinician time for manual segmentation and reasoning chain generation.

Result: Established ReasonKnee-Bench for evaluating VLM localization and diagnostic accuracy. Benchmarked five state-of-the-art VLMs to provide baseline performance. The dataset enables assessment of grounding and severity assessment capabilities across anatomical regions.

Conclusion: 3DReasonKnee serves as a repository of orthopedic surgeons’ diagnostic expertise and provides a vital testbed for advancing multimodal medical AI systems towards 3D, clinically aligned, localized decision-making capabilities.

Abstract: Current Vision-Language Models (VLMs) struggle to ground anatomical regions in 3D medical images and reason about them in a step-by-step manner, a key requirement of real-world diagnostic assessment. This ability is essential for aligning model outputs with the diagnostic workflows clinicians use in practice, enabling trustworthy clinician-AI collaboration. Existing 3D datasets provide localization labels, but none support this “grounded reasoning” ability. To address this gap, we introduce 3DReasonKnee, the first 3D grounded reasoning dataset for medical images, which provides 494k high-quality quintuples derived from 7,970 3D knee MRI volumes. Each quintuple includes: (1) the 3D MRI volume, (2) a diagnostic question targeting a specific anatomical region (3) a 3D bounding box localizing the relevant anatomical structures, (4) clinician-generated diagnostic reasoning steps that explicitly detail the 3D reasoning process, and (5) structured severity assessments for the relevant anatomical region. The creation and validation of 3DReasonKnee, involving over 450 hours of expert clinician time for manually segmenting MRIs and generating reasoning chains, ensures its superior quality and clinical relevance. We establish ReasonKnee-Bench to evaluate localization and diagnostic accuracy, providing insight into VLM ability to perform grounding and severity assessment across anatomical regions and diagnostic inquiries. We benchmark five state-of-the-art VLMs, providing baseline performance for ReasonKnee-Bench. By providing this unique resource of expert-annotated 3D reasoning pathways, 3DReasonKnee serves as a repository of orthopedic surgeons’ diagnostic expertise and offers a vital testbed for advancing multimodal medical AI systems towards 3D, clinically aligned, localized decision-making capabilities. The dataset can be found in: https://huggingface.co/datasets/rajpurkarlab/3DReasonKnee

Method: Uses multi-view thermal polarimetric imaging in long-wave infrared (LWIR) spectrum, formulating a general theory of polarization observation that eliminates ambiguities present in visible light polarization.

Result: Effectively reconstructs fine details in transparent, translucent, and heterogeneous objects, outperforming existing state-of-the-art techniques.

Conclusion: Thermal polarization-based 3D reconstruction provides a robust solution independent of illumination and material constraints, enabling detailed shape recovery for challenging object types.

Abstract: This paper introduces a novel method for detailed 3D shape reconstruction utilizing thermal polarization cues. Unlike state-of-the-art methods, the proposed approach is independent of illumination and material properties. In this paper, we formulate a general theory of polarization observation and show that long-wave infrared (LWIR) polarimetric imaging is free from the ambiguities that affect visible polarization analyses. Subsequently, we propose a method for recovering detailed 3D shapes using multi-view thermal polarimetric images. Experimental results demonstrate that our approach effectively reconstructs fine details in transparent, translucent, and heterogeneous objects, outperforming existing techniques.

Method: Connects V-JEPA2 video embeddings to frozen Stable Audio Open DiT text-to-audio model by inserting compact video cross-attention after existing text cross-attention, with video token pooling for memory efficiency and training stability.

Result: Achieves competitive temporal and semantic alignment on video-audio benchmarks with significantly fewer trainable parameters than multi-modal systems, while preserving prompt-driven controllability and modular architecture.

Conclusion: The lightweight bridge approach effectively learns audio-video synchronization without retraining audio priors, offering production-friendly modularity and potential extension to other audio modalities beyond Foley.

Abstract: Foley Control is a lightweight approach to video-guided Foley that keeps pretrained single-modality models frozen and learns only a small cross-attention bridge between them. We connect V-JEPA2 video embeddings to a frozen Stable Audio Open DiT text-to-audio (T2A) model by inserting compact video cross-attention after the model’s existing text cross-attention, so prompts set global semantics while video refines timing and local dynamics. The frozen backbones retain strong marginals (video; audio given text) and the bridge learns the audio-video dependency needed for synchronization – without retraining the audio prior. To cut memory and stabilize training, we pool video tokens before conditioning. On curated video-audio benchmarks, Foley Control delivers competitive temporal and semantic alignment with far fewer trainable parameters than recent multi-modal systems, while preserving prompt-driven controllability and production-friendly modularity (swap/upgrade encoders or the T2A backbone without end-to-end retraining). Although we focus on Video-to-Foley, the same bridge design can potentially extend to other audio modalities (e.g., speech).

Method: VESSA uses a self-distillation paradigm with multi-view object-centric videos. It carefully tunes prediction heads and deploys parameter-efficient adaptation techniques to prevent forgetting pretrained knowledge. The method learns from different frames in object-centric videos to capture robustness to varied conditions.

Result: Comprehensive experiments with 3 vision foundation models on 2 datasets show VESSA consistently improves downstream classification performance compared to base models and previous adaptation methods.

Conclusion: VESSA successfully enables self-supervised adaptation of vision foundation models to new domains using only unlabeled object-centric videos, addressing the challenge of distribution shifts without requiring annotations.

Abstract: Foundation models have advanced computer vision by enabling strong performance across diverse tasks through large-scale pretraining and supervised fine-tuning. However, they may underperform in domains with distribution shifts and scarce labels, where supervised fine-tuning may be infeasible. While continued self-supervised learning for model adaptation is common for generative language models, this strategy has not proven effective for vision-centric encoder models. To address this challenge, we introduce a novel formulation of self-supervised fine-tuning for vision foundation models, where the model is adapted to a new domain without requiring annotations, leveraging only short multi-view object-centric videos. Our method is referred to as VESSA: Video-based objEct-centric Self-Supervised Adaptation for visual foundation models. VESSA’s training technique is based on a self-distillation paradigm, where it is critical to carefully tune prediction heads and deploy parameter-efficient adaptation techniques - otherwise, the model may quickly forget its pretrained knowledge and reach a degraded state. VESSA benefits significantly from multi-view object observations sourced from different frames in an object-centric video, efficiently learning robustness to varied capture conditions, without the need of annotations. Through comprehensive experiments with 3 vision foundation models on 2 datasets, VESSA demonstrates consistent improvements in downstream classification tasks, compared to the base models and previous adaptation methods. Code is publicly available at https://github.com/jesimonbarreto/VESSA.

Method: Uses low-light image enhancement and background removal guided by open-vocabulary detection with foundation models, building on current SOTA baselines to reduce domain shift and suppress false positives from raw SAM outputs.

Result: Significantly boosts detection accuracy on real-world industrial bin-picking benchmarks from BOP while incurring negligible inference overhead.

Conclusion: The proposed method is effective and practical for improving 2D detection in industrial settings, enabling more reliable detections for downstream pose estimation tasks.

Abstract: Accurate 6D pose estimation is essential for robotic manipulation in industrial environments. Existing pipelines typically rely on off-the-shelf object detectors followed by cropping and pose refinement, but their performance degrades under challenging conditions such as clutter, poor lighting, and complex backgrounds, making detection the critical bottleneck. In this work, we introduce a standardized and plug-in pipeline for 2D detection of unseen objects in industrial settings. Based on current SOTA baselines, our approach reduces domain shift and background artifacts through low-light image enhancement and background removal guided by open-vocabulary detection with foundation models. This design suppresses the false positives prevalent in raw SAM outputs, yielding more reliable detections for downstream pose estimation. Extensive experiments on real-world industrial bin-picking benchmarks from BOP demonstrate that our method significantly boosts detection accuracy while incurring negligible inference overhead, showing the effectiveness and practicality of the proposed method.

Method: Proposes a carefully designed multi-stage training methodology that progressively trains LLMs to understand both visual and speech information.

Result: Preserves strong vision-language capacity while enabling efficient speech-to-speech dialogue without separate ASR/TTS modules, significantly accelerating end-to-end response speed.

Conclusion: The model demonstrates strong visual and speech capabilities across image, video, and speech benchmarks, enabling near real-time vision and speech interaction.

Abstract: Recent Multimodal Large Language Models (MLLMs) have typically focused on integrating visual and textual modalities, with less emphasis placed on the role of speech in enhancing interaction. However, speech plays a crucial role in multimodal dialogue systems, and implementing high-performance in both vision and speech tasks remains a significant challenge due to the fundamental modality differences. In this paper, we propose a carefully designed multi-stage training methodology that progressively trains LLM to understand both visual and speech information, ultimately enabling fluent vision and speech interaction. Our approach not only preserves strong vision-language capacity, but also enables efficient speech-to-speech dialogue capabilities without separate ASR and TTS modules, significantly accelerating multimodal end-to-end response speed. By comparing our method against state-of-the-art counterparts across benchmarks for image, video, and speech tasks, we demonstrate that our model is equipped with both strong visual and speech capabilities, making near real-time vision and speech interaction. Code has been released at https://github.com/VITA-MLLM/VITA.

Method: Developed a hybrid deep learning framework using YOLOv11 models trained on new datasets from the 2023 Turkey Earthquakes. The approach includes fine-tuning, data augmentation, automated classification of building components, and damage level detection through multiple trained models.

Result: The framework successfully detects cracking, spalling damage, exposed steel bars, and distinguishes different structural damage levels automatically from input images.

Conclusion: Rapid and automated damage detection following disasters is achievable across diverse damage contexts by utilizing image data collection, annotation, and deep learning approaches.

Abstract: Timely assessment of integrity of structures after seismic events is crucial for public safety and emergency response. This study focuses on assessing the structural damage conditions using deep learning methods to detect exposed steel reinforcement in concrete buildings and bridges after large earthquakes. Steel bars are typically exposed after concrete spalling or large flexural or shear cracks. The amount and distribution of exposed steel reinforcement is an indication of structural damage and degradation. To automatically detect exposed steel bars, new datasets of images collected after the 2023 Turkey Earthquakes were labeled to represent a wide variety of damaged concrete structures. The proposed method builds upon a deep learning framework, enhanced with fine-tuning, data augmentation, and testing on public datasets. An automated classification framework is developed that can be used to identify inside/outside buildings and structural components. Then, a YOLOv11 (You Only Look Once) model is trained to detect cracking and spalling damage and exposed bars. Another YOLO model is finetuned to distinguish different categories of structural damage levels. All these trained models are used to create a hybrid framework to automatically and reliably determine the damage levels from input images. This research demonstrates that rapid and automated damage detection following disasters is achievable across diverse damage contexts by utilizing image data collection, annotation, and deep learning approaches.

Method: Leverages VLM reasoning to generate rich 2D scene graphs, then grounds them in 3D using depth information. Creates nodes with open-vocabulary objects, features, 3D locations, and semantic context, with edges capturing spatial/semantic relations and inter-object distances.

Result: Experiments on Replica and HM3D datasets show ZING-3D effectively captures spatial and relational knowledge without task-specific training, demonstrating zero-shot capability.

Conclusion: ZING-3D provides a practical solution for 3D scene understanding in robotics by combining foundation model knowledge with geometric grounding and incremental updates, enabling rich semantic representations without specialized training.

Abstract: Understanding and reasoning about complex 3D environments requires structured scene representations that capture not only objects but also their semantic and spatial relationships. While recent works on 3D scene graph generation have leveraged pretrained VLMs without task-specific fine-tuning, they are largely confined to single-view settings, fail to support incremental updates as new observations arrive and lack explicit geometric grounding in 3D space, all of which are essential for embodied scenarios. In this paper, we propose, ZING-3D, a framework that leverages the vast knowledge of pretrained foundation models to enable open-vocabulary recognition and generate a rich semantic representation of the scene in a zero-shot manner while also enabling incremental updates and geometric grounding in 3D space, making it suitable for downstream robotics applications. Our approach leverages VLM reasoning to generate a rich 2D scene graph, which is grounded in 3D using depth information. Nodes represent open-vocabulary objects with features, 3D locations, and semantic context, while edges capture spatial and semantic relations with inter-object distances. Our experiments on scenes from the Replica and HM3D dataset show that ZING-3D is effective at capturing spatial and relational knowledge without the need of task-specific training.

Method: Proposes WaveSeg decoder with: 1) Learning high-frequency components as explicit priors for boundary details, 2) Dual Domain Operation (DDO) for multi-scale fusion, 3) Spectrum Decomposition Attention (SDA) block using Mamba’s linear-complexity long-range modeling, 4) Reparameterized convolutions for low-frequency semantic preservation, 5) Residual-guided fusion for multi-scale feature integration.

Result: Extensive experiments on standard benchmarks show WaveSeg consistently outperforms state-of-the-art approaches both quantitatively and qualitatively, achieving efficient and precise segmentation.

Conclusion: WaveSeg demonstrates that leveraging wavelet-domain frequency priors with Mamba-based attention enables superior semantic segmentation performance by effectively balancing semantic context and fine-grained detail preservation.

Abstract: While recent semantic segmentation networks heavily rely on powerful pretrained encoders, most employ simplistic decoders, leading to suboptimal trade-offs between semantic context and fine-grained detail preservation. To address this, we propose a novel decoder architecture, WaveSeg, which jointly optimizes feature refinement in spatial and wavelet domains. Specifically, high-frequency components are first learned from input images as explicit priors to reinforce boundary details at early stages. A multi-scale fusion mechanism, Dual Domain Operation (DDO), is then applied, and the novel Spectrum Decomposition Attention (SDA) block is proposed, which is developed to leverage Mamba’s linear-complexity long-range modeling to enhance high-frequency structural details. Meanwhile, reparameterized convolutions are applied to preserve low-frequency semantic integrity in the wavelet domain. Finally, a residual-guided fusion integrates multi-scale features with boundary-aware representations at native resolution, producing semantically and structurally rich feature maps. Extensive experiments on standard benchmarks demonstrate that WaveSeg, leveraging wavelet-domain frequency prior with Mamba-based attention, consistently outperforms state-of-the-art approaches both quantitatively and qualitatively, achieving efficient and precise segmentation.

Method: Integrates expert-derived textual concepts from medical sources into a Contrastive Language-Image Pre-training-based vision-language model, using LLM-generated prompts reviewed by experts and encoded with QuiltNet.

Result: Achieved 83.9% accuracy, 86.6% precision, and 87.6% specificity, outperforming VGG-19, ResNet-18, and ResNet-50 models.

Conclusion: Multi-modal learning incorporating expert knowledge shows superior performance and produces more clinically relevant outputs for histopathology tasks.

Abstract: Hirschsprung’s disease is defined as the congenital absence of ganglion cells in some segment(s) of the colon. The muscle cannot make coordinated movements to propel stool in that section, most commonly leading to obstruction. The diagnosis and treatment for this disease require a clear identification of different region(s) of the myenteric plexus, where ganglion cells should be present, on the microscopic view of the tissue slide. While deep learning approaches, such as Convolutional Neural Networks, have performed very well in this task, they are often treated as black boxes, with minimal understanding gained from them, and may not conform to how a physician makes decisions. In this study, we propose a novel framework that integrates expert-derived textual concepts into a Contrastive Language-Image Pre-training-based vision-language model to guide plexus classification. Using prompts derived from expert sources (e.g., medical textbooks and papers) generated by large language models and reviewed by our team before being encoded with QuiltNet, our approach aligns clinically relevant semantic cues with visual features. Experimental results show that the proposed model demonstrated superior discriminative capability across different classification metrics as it outperformed CNN-based models, including VGG-19, ResNet-18, and ResNet-50; achieving an accuracy of 83.9%, a precision of 86.6%, and a specificity of 87.6%. These findings highlight the potential of multi-modal learning in histopathology and underscore the value of incorporating expert knowledge for more clinically relevant model outputs.

Method: Extends Retinex model to histogram domain using histogram location/count matrices, constructs two-level optimization model, and provides iterative formulas for illumination and reflectance histograms.

Result: Outperforms existing methods in visibility and performance metrics, executes in 1.86 seconds on 1000*664 images (minimum 6.67 seconds faster than alternatives).

Conclusion: HistRetinex provides an efficient and effective solution for fast low-light image enhancement with superior performance and significant speed advantages.

Abstract: Retinex-based low-light image enhancement methods are widely used due to their excellent performance. However, most of them are time-consuming for large-sized images. This paper extends the Retinex model from the spatial domain to the histogram domain, and proposes a novel histogram-based Retinex model for fast low-light image enhancement, named HistRetinex. Firstly, we define the histogram location matrix and the histogram count matrix, which establish the relationship among histograms of the illumination, reflectance and the low-light image. Secondly, based on the prior information and the histogram-based Retinex model, we construct a novel two-level optimization model. Through solving the optimization model, we give the iterative formulas of the illumination histogram and the reflectance histogram, respectively. Finally, we enhance the low-light image through matching its histogram with the one provided by HistRetinex. Experimental results demonstrate that the HistRetinex outperforms existing enhancement methods in both visibility and performance metrics, while executing 1.86 seconds on 1000*664 resolution images, achieving a minimum time saving of 6.67 seconds.

Method: Created CLEVR-AVR benchmark for multi-round interactive environments, developed AVR-152k dataset with Chain-of-Thought annotations for uncertainty identification and action selection, and built PhysVLM-AVR model that operates in higher-order Markov Decision Process.

Result: PhysVLM-AVR achieves state-of-the-art performance on CLEVR-AVR, embodied reasoning (OpenEQA, RoboVQA), and passive visual reasoning (GeoMath, Geometry30K). Analysis shows current embodied MLLMs struggle with active information acquisition and integration despite detecting incompleteness.

Conclusion: Active visual reasoning requires fundamental capabilities beyond current MLLMs, including active information gathering, multi-step observation integration, and dynamic decision adjustment. The AVR framework and PhysVLM-AVR demonstrate significant advances in bridging this gap.

Abstract: Visual reasoning in multimodal large language models (MLLMs) has primarily been studied in static, fully observable settings, limiting their effectiveness in real-world environments where information is often incomplete due to occlusion or limited field of view. Humans, in contrast, actively explore and interact with their environment-moving, examining, and manipulating objects-to gather information through a closed-loop process integrating perception, reasoning, and action. Inspired by this human capability, we introduce the Active Visual Reasoning (AVR) task, extending visual reasoning to partially observable, interactive environments. AVR necessitates agents to: (1) actively acquire information via sequential physical actions, (2) integrate observations across multiple steps for coherent reasoning, and (3) dynamically adjust decisions based on evolving visual feedback. To rigorously evaluate AVR, we introduce CLEVR-AVR, a simulation benchmark featuring multi-round interactive environments designed to assess both reasoning correctness and information-gathering efficiency. We present AVR-152k, a large-scale dataset that offers rich Chain-of-Thought (CoT) annotations detailing iterative reasoning for uncertainty identification, action-conditioned information gain prediction, and information-maximizing action selection, crucial for training agents in a higher-order Markov Decision Process. Building on this, we develop PhysVLM-AVR, an MLLM achieving state-of-the-art performance on CLEVR-AVR, embodied reasoning (OpenEQA, RoboVQA), and passive visual reasoning (GeoMath, Geometry30K). Our analysis also reveals that current embodied MLLMs, despite detecting information incompleteness, struggle to actively acquire and integrate new information through interaction, highlighting a fundamental gap in active reasoning capabilities.

Method: Multi-resolution NDT alignment with point-to-plane ICP, height normalization, uncertainty-aware detection using registration covariance and surface roughness, class-constrained bipartite assignment with dummy augmentation for splits/merges, and tiled processing with instance-level decision fusion.

Result: Achieved 95.2% accuracy, 90.4% mF1, and 82.6% mIoU on 15 Subiaco blocks, outperforming Triplet KPConv by 0.2% in accuracy, 0.2% in mF1, and 0.8% in mIoU, with largest improvement on Decreased class (74.8% IoU, +7.6 points).

Conclusion: The proposed uncertainty-aware, object-centric pipeline effectively handles real-world challenges in city-scale LiDAR change detection, particularly excelling at detecting object decreases while maintaining class consistency and robustness to partial overlaps.

Abstract: High-definition 3D city maps underpin smart transportation, digital twins, and autonomous driving, where object level change detection across bi temporal LiDAR enables HD map maintenance, construction monitoring, and reliable localization. Classical DSM differencing and image based methods are sensitive to small vertical bias, ground slope, and viewpoint mismatch and yield cellwise outputs without object identity. Point based neural models and voxel encodings demand large memory, assume near perfect pre alignment, degrade thin structures, and seldom enforce class consistent association, which leaves split or merge cases unresolved and ignores uncertainty. We propose an object centric, uncertainty aware pipeline for city scale LiDAR that aligns epochs with multi resolution NDT followed by point to plane ICP, normalizes height, and derives a per location level of detection from registration covariance and surface roughness to calibrate decisions and suppress spurious changes. Geometry only proxies seed cross epoch associations that are refined by semantic and instance segmentation and a class constrained bipartite assignment with augmented dummies to handle splits and merges while preserving per class counts. Tiled processing bounds memory without eroding narrow ground changes, and instance level decisions combine 3D overlap, normal direction displacement, and height and volume differences with a histogram distance, all gated by the local level of detection to remain stable under partial overlap and sampling variation. On 15 representative Subiaco blocks the method attains 95.2% accuracy, 90.4% mF1, and 82.6% mIoU, exceeding Triplet KPConv by 0.2 percentage points in accuracy, 0.2 in mF1, and 0.8 in mIoU, with the largest gain on Decreased where IoU reaches 74.8% and improves by 7.6 points.

Method: Uses a lightweight dynamic mixed local priors extractor with heterogeneous convolutions and gating network to capture diverse local priors, combined with a bi-directional interaction adapter using cosine-aligned deformable attention and channel-oriented adaptive scale enhancement.

Result: Extensive experiments show superior segmentation performance compared to 31 state-of-the-art methods and adaptability to multiple binary object segmentation tasks.

Conclusion: Controllable-LPMoE provides an efficient fine-tuning approach that enhances fine-grained perception for segmentation tasks while significantly reducing computational overhead.

Abstract: Large-scale foundation models provide powerful feature representations for downstream object segmentation tasks. However, when adapted to specific tasks through the full-parameter fine-tuning, the enormous parameters being updated often results in significant computational overhead, creating a bottleneck in training efficiency. Although existing methods attempt to fine-tune frozen models by directly embedding trainable prompts, these prompts lack inherent semantic priors, limiting the adaptability of large-scale models. In this paper, we propose a novel dynamic priors-based fine-tuning paradigm with fewer trainable parameters, dubbed Controllable-LPMoE, which adaptively modulates frozen foundation models by dynamically controlling local priors to enhance fine-grained perception for specific segmentation tasks. More specifically, we construct a lightweight dynamic mixed local priors extractor that captures diverse local priors from input images through heterogeneous convolutions while employing a gating network to dynamically output expert priors required for the subsequent fine-tuning. Furthermore, we design a bi-directional interaction adapter that employs cosine-aligned deformable attention and channel-oriented adaptive scale enhancement to interact and restructure between frozen and trainable features, achieving efficient fine-tuning. Extensive experiments validate the superiority of our \href{https://github.com/CSYSI/Controllable-LPMoE} {Controllable-LPMoE} approach, demonstrating excellent segmentation performance compared to 31 state-of-the-art (SOTA) methods and adaptability to multiple binary object segmentation tasks.

Method: Leverage image editing models to create targeted changes that alter safety labels while preserving safety-irrelevant details, generating counterfactual pairs that flip safety labels.

Result: Created a benchmark with 3,020 SafetyPair images across 9 safety categories. The framework serves as effective evaluation data and improves sample efficiency for training lightweight guard models.

Conclusion: SafetyPairs provides the first systematic resource for studying fine-grained image safety distinctions and highlights weaknesses in vision-language models’ ability to distinguish subtly different images.

Abstract: What exactly makes a particular image unsafe? Systematically differentiating between benign and problematic images is a challenging problem, as subtle changes to an image, such as an insulting gesture or symbol, can drastically alter its safety implications. However, existing image safety datasets are coarse and ambiguous, offering only broad safety labels without isolating the specific features that drive these differences. We introduce SafetyPairs, a scalable framework for generating counterfactual pairs of images, that differ only in the features relevant to the given safety policy, thus flipping their safety label. By leveraging image editing models, we make targeted changes to images that alter their safety labels while leaving safety-irrelevant details unchanged. Using SafetyPairs, we construct a new safety benchmark, which serves as a powerful source of evaluation data that highlights weaknesses in vision-language models’ abilities to distinguish between subtly different images. Beyond evaluation, we find our pipeline serves as an effective data augmentation strategy that improves the sample efficiency of training lightweight guard models. We release a benchmark containing over 3,020 SafetyPair images spanning a diverse taxonomy of 9 safety categories, providing the first systematic resource for studying fine-grained image safety distinctions.

Method: Two key components: (1) Noise-Injected Exploration Policy that perturbs visual inputs with Gaussian noise, and (2) Bayesian Advantage Estimation that formulates advantage estimation as Bayesian inference using noise level as prior and trajectory reward as likelihood.

Result: Experiments show NoisyGRPO substantially improves generalization and robustness on CoT quality, general capability, and hallucination benchmarks, especially with small-scale MLLMs like Qwen2.5-VL 3B.

Conclusion: NoisyGRPO provides an effective RL framework that enhances MLLMs’ CoT reasoning through systematic noise injection and principled Bayesian advantage estimation, improving generalization capabilities.

Abstract: Reinforcement learning (RL) has shown promise in enhancing the general Chain-of-Thought (CoT) reasoning capabilities of multimodal large language models (MLLMs). However, when applied to improve general CoT reasoning, existing RL frameworks often struggle to generalize beyond the training distribution. To address this, we propose NoisyGRPO, a systematic multimodal RL framework that introduces controllable noise into visual inputs for enhanced exploration and explicitly models the advantage estimation process via a Bayesian framework. Specifically, NoisyGRPO improves RL training by: (1) \textbf{Noise-Injected Exploration Policy}: Perturbing visual inputs with Gaussian noise to encourage exploration across a wider range of visual scenarios; and (2) \textbf{Bayesian Advantage Estimation}: Formulating advantage estimation as a principled Bayesian inference problem, where the injected noise level serves as a prior and the observed trajectory reward as the likelihood. This Bayesian modeling fuses both sources of information to compute a robust posterior estimate of trajectory advantage, effectively guiding MLLMs to prefer visually grounded trajectories over noisy ones. Experiments on standard CoT quality, general capability, and hallucination benchmarks demonstrate that NoisyGRPO substantially improves generalization and robustness, especially in RL settings with small-scale MLLMs such as Qwen2.5-VL 3B. The project page is available at \href{https://artanic30.github.io/project_pages/NoisyGRPO/}{\texttt{https://artanic30.github.io/project_pages/NoisyGRPO}}.

Method: Used a cascaded synthesizer based on Cycle-Consistent Generative Adversarial Networks (CycleGAN) trained on 410 paired CTPA and NCCT scans from three centers, with 249 for training/validation and 161 for testing.

Result: Achieved best performance compared to SOTA methods with MAE: 165.12, PSNR: 20.27, SSIM: 0.98 on test set. Downstream tasks showed significant improvements in pulmonary vessel segmentation (Dice: 0.70) and vascular quantification (ICC: 0.81 vs 0.70 for NCCT).

Conclusion: The DCCTPA method effectively enhances vascular structures without contrast agents, providing superior image quality and enabling accurate pulmonary vessel analysis, particularly benefiting small vessel visualization.

Abstract: Computed Tomography Pulmonary Angiography (CTPA) is the reference standard for diagnosing pulmonary vascular diseases such as Pulmonary Embolism (PE) and Chronic Thromboembolic Pulmonary Hypertension (CTEPH). However, its reliance on iodinated contrast agents poses risks including nephrotoxicity and allergic reactions, particularly in high-risk patients. This study proposes a method to generate Digital Contrast CTPA (DCCTPA) from Non-Contrast CT (NCCT) scans using a cascaded synthesizer based on Cycle-Consistent Generative Adversarial Networks (CycleGAN). Totally retrospective 410 paired CTPA and NCCT scans were obtained from three centers. The model was trained and validated internally on 249 paired images. Extra dataset that comprising 161 paired images was as test set for model generalization evaluation and downstream clinical tasks validation. Compared with state-of-the-art (SOTA) methods, the proposed method achieved the best comprehensive performance by evaluating quantitative metrics (For validation, MAE: 156.28, PSNR: 20.71 and SSIM: 0.98; For test, MAE: 165.12, PSNR: 20.27 and SSIM: 0.98) and qualitative visualization, demonstrating valid vessel enhancement, superior image fidelity and structural preservation. The approach was further applied to downstream tasks of pulmonary vessel segmentation and vascular quantification. On the test set, the average Dice, clDice, and clRecall of artery and vein pulmonary segmentation was 0.70, 0.71, 0.73 and 0.70, 0.72, 0.75 respectively, all markedly improved compared with NCCT inputs.@ Inter-class Correlation Coefficient (ICC) for vessel volume between DCCTPA and CTPA was significantly better than that between NCCT and CTPA (Average ICC : 0.81 vs 0.70), indicating effective vascular enhancement in DCCTPA, especially for small vessels.

Method: Introduces SIG: a grid-based schema that encodes object layouts, inter-object relations, and physically grounded priors as a complementary channel to text for foundation-model reasoning.

Result: SIG yields consistently larger, more stable, and more comprehensive gains across all VSI metrics compared to VQA-only representations in few-shot learning with state-of-the-art multimodal LLMs.

Conclusion: SIG shows promise as a data-labeling and training schema for learning VSI, with SIGBench providing a benchmark for both grid-based machine VSI tasks and human-like VSI tasks in autonomous driving.

Abstract: How to integrate and verify spatial intelligence in foundation models remains an open challenge. Current practice often proxies Visual-Spatial Intelligence (VSI) with purely textual prompts and VQA-style scoring, which obscures geometry, invites linguistic shortcuts, and weakens attribution to genuinely spatial skills. We introduce Spatial Intelligence Grid (SIG): a structured, grid-based schema that explicitly encodes object layouts, inter-object relations, and physically grounded priors. As a complementary channel to text, SIG provides a faithful, compositional representation of scene structure for foundation-model reasoning. Building on SIG, we derive SIG-informed evaluation metrics that quantify a model’s intrinsic VSI, which separates spatial capability from language priors. In few-shot in-context learning with state-of-the-art multimodal LLMs (e.g. GPT- and Gemini-family models), SIG yields consistently larger, more stable, and more comprehensive gains across all VSI metrics compared to VQA-only representations, indicating its promise as a data-labeling and training schema for learning VSI. We also release SIGBench, a benchmark of 1.4K driving frames annotated with ground-truth SIG labels and human gaze traces, supporting both grid-based machine VSI tasks and attention-driven, human-like VSI tasks in autonomous-driving scenarios.

Method: BFM divides the generative trajectory into multiple temporal segments, each handled by smaller specialized velocity blocks. It introduces Semantic Feature Guidance to condition blocks on pretrained representations and Feature Residual Approximation to reduce inference cost while maintaining quality.

Result: On ImageNet 256x256, BFM achieves 2.1x to 4.9x acceleration in inference complexity while maintaining comparable generation performance, establishing a superior Pareto frontier over existing Flow Matching methods.

Conclusion: BFM provides an efficient and high-quality alternative to standard Flow Matching by leveraging specialized blockwise architecture with semantic guidance, significantly reducing computational costs without sacrificing generation fidelity.

Abstract: Recently, Flow Matching models have pushed the boundaries of high-fidelity data generation across a wide range of domains. It typically employs a single large network to learn the entire generative trajectory from noise to data. Despite their effectiveness, this design struggles to capture distinct signal characteristics across timesteps simultaneously and incurs substantial inference costs due to the iterative evaluation of the entire model. To address these limitations, we propose Blockwise Flow Matching (BFM), a novel framework that partitions the generative trajectory into multiple temporal segments, each modeled by smaller but specialized velocity blocks. This blockwise design enables each block to specialize effectively in its designated interval, improving inference efficiency and sample quality. To further enhance generation fidelity, we introduce a Semantic Feature Guidance module that explicitly conditions velocity blocks on semantically rich features aligned with pretrained representations. Additionally, we propose a lightweight Feature Residual Approximation strategy that preserves semantic quality while significantly reducing inference cost. Extensive experiments on ImageNet 256x256 demonstrate that BFM establishes a substantially improved Pareto frontier over existing Flow Matching methods, achieving 2.1x to 4.9x accelerations in inference complexity at comparable generation performance. Code is available at https://github.com/mlvlab/BFM.

Method: TokenCLIP expands the token-agnostic textual space into orthogonal subspaces, dynamically assigns each visual token to subspace combinations via optimal transport based on semantic affinity, and applies top-k masking to specialize subspaces for distinct visual regions.

Result: Extensive experiments demonstrate the superiority of TokenCLIP over existing methods in anomaly detection performance.

Conclusion: TokenCLIP’s dynamic token-wise alignment approach effectively captures fine-grained anomaly semantics and provides a more accurate and efficient framework for zero-shot anomaly detection.

Abstract: Adapting CLIP for anomaly detection on unseen objects has shown strong potential in a zero-shot manner. However, existing methods typically rely on a single textual space to align with visual semantics across diverse objects and domains. The indiscriminate alignment hinders the model from accurately capturing varied anomaly semantics. We propose TokenCLIP, a token-wise adaptation framework that enables dynamic alignment between visual and learnable textual spaces for fine-grained anomaly learning. Rather than mapping all visual tokens to a single, token-agnostic textual space, TokenCLIP aligns each token with a customized textual subspace that represents its visual characteristics. Explicitly assigning a unique learnable textual space to each token is computationally intractable and prone to insufficient optimization. We instead expand the token-agnostic textual space into a set of orthogonal subspaces, and then dynamically assign each token to a subspace combination guided by semantic affinity, which jointly supports customized and efficient token-wise adaptation. To this end, we formulate dynamic alignment as an optimal transport problem, where all visual tokens in an image are transported to textual subspaces based on semantic similarity. The transport constraints of OT ensure sufficient optimization across subspaces and encourage them to focus on different semantics. Solving the problem yields a transport plan that adaptively assigns each token to semantically relevant subspaces. A top-k masking is then applied to sparsify the plan and specialize subspaces for distinct visual regions. Extensive experiments demonstrate the superiority of TokenCLIP.

Method: Uses graph formulation to represent VQA samples, then expands static benchmarks by integrating multimodal knowledge through reconstructing questions with re-selected visual information and expanding questions with external textual knowledge.

Result: KBE enables difficulty-controllable evaluation by adjusting question exploration degree, alleviates data contamination and saturation risks, and provides more comprehensive MLLM capability assessment.

Conclusion: The proposed KBE framework offers a more reliable and dynamic evaluation approach for multimodal LLMs compared to traditional static benchmarks.

Abstract: The rapid progress of multimodal large language models (MLLMs) calls for more reliable evaluation protocols. Existing static benchmarks suffer from the potential risk of data contamination and saturation, leading to inflated or misleading performance evaluations. To address these issues, we first apply Graph formulation to represent a static or dynamic VQA sample. With the formulation, we propose Knowledge-enhanced Benchmark Evolution(KBE), a dynamic multimodal evaluation framework. KBE first analyzes the original static benchmark, then expands it by integrating multimodal knowledge, transforming the static benchmark into a controllable, dynamic evolving version. Crucially, KBE can both reconstruct questions by Re-selecting visual information in the original image and expand existing questions with external textual knowledge. It enables difficulty-controllable evaluation by adjusting the degree of question exploration. Extensive experiments demonstrate that KBE alleviates the risk of data contamination, data saturation, and provides a more comprehensive assessment of MLLM capabilities.

Method: Train four basic models with weighted sum of ArcFace and Circle loss, apply TTA and ensemble, and propose a new reranking method combining diffusion and k-reciprocal reranking.

Result: Achieved 0.81219 mAP@100 on public leaderboard and 0.81191 mAP@100 on private leaderboard.

Conclusion: The proposed ensemble approach with novel reranking method proved effective for fine-grained food image retrieval, securing 3rd place in the competition.

Abstract: This paper introduces the 3rd place solution to the ICCV LargeFineFoodAI Retrieval Competition on Kaggle. Four basic models are independently trained with the weighted sum of ArcFace and Circle loss, then TTA and Ensemble are successively applied to improve feature representation ability. In addition, a new reranking method for retrieval is proposed based on diffusion and k-reciprocal reranking. Finally, our method scored 0.81219 and 0.81191 mAP@100 on the public and private leaderboard, respectively.

Method: Used a combination of Arcface loss and Circle loss functions, trained models with carefully tuned configurations, and employed model ensembling techniques.

Result: The solution achieved 3rd place in the LargeFineFoodAI-ICCV Workshop-Recognition challenge on Kaggle.

Conclusion: Proper combination of Arcface and Circle loss functions can improve performance in fine-grained food recognition tasks.

Abstract: Food analysis is becoming a hot topic in health area, in which fine-grained food recognition task plays an important role. In this paper, we describe the details of our solution to the LargeFineFoodAI-ICCV Workshop-Recognition challenge held on Kaggle. We find a proper combination of Arcface loss[1] and Circle loss[9] can bring improvement to the performance. With Arcface and the combined loss, model was trained with carefully tuned configurations and ensembled to get the final results. Our solution won the 3rd place in the competition.

Method: Proposes iSM framework with four key improvements: Intrinsic Guidance for dynamic control, Multi-Level Wavelet Loss to mitigate frequency bias, Scaling Optimal Transport (sOT) for straighter generative paths, and Twin EMA strategy for training stability and self-consistency.

Result: Extensive experiments on ImageNet 256x256 demonstrate substantial FID improvements over baseline shortcut models across one-step, few-step, and multi-step generation, making shortcut models competitive.

Conclusion: The iSM framework systematically resolves the core limitations of shortcut models, transforming them into a viable and competitive class of generative models capable of high-quality one-step, few-step, and multi-step sampling from a single network.

Abstract: Shortcut models represent a promising, non-adversarial paradigm for generative modeling, uniquely supporting one-step, few-step, and multi-step sampling from a single trained network. However, their widespread adoption has been stymied by critical performance bottlenecks. This paper tackles the five core issues that held shortcut models back: (1) the hidden flaw of compounding guidance, which we are the first to formalize, causing severe image artifacts; (2) inflexible fixed guidance that restricts inference-time control; (3) a pervasive frequency bias driven by a reliance on low-level distances in the direct domain, which biases reconstructions toward low frequencies; (4) divergent self-consistency arising from a conflict with EMA training; and (5) curvy flow trajectories that impede convergence. To address these challenges, we introduce iSM, a unified training framework that systematically resolves each limitation. Our framework is built on four key improvements: Intrinsic Guidance provides explicit, dynamic control over guidance strength, resolving both compounding guidance and inflexibility. A Multi-Level Wavelet Loss mitigates frequency bias to restore high-frequency details. Scaling Optimal Transport (sOT) reduces training variance and learns straighter, more stable generative paths. Finally, a Twin EMA strategy reconciles training stability with self-consistency. Extensive experiments on ImageNet 256 x 256 demonstrate that our approach yields substantial FID improvements over baseline shortcut models across one-step, few-step, and multi-step generation, making shortcut models a viable and competitive class of generative models.

Method: Uses Discrete Diffusion Models with three innovations: 1) Decoupled Training and Hybrid Inference separating topology sculpting and shape refinement stages, 2) Improved Hourglass Architecture with bidirectional attention and face-vertex-sequence level Rotational Positional Embeddings, 3) Connection Loss as topological constraint.

Result: Generates high-quality 3D artist-style meshes with up to 10,000 faces at spatial resolution of 1024³, demonstrating superior performance on complex datasets.

Conclusion: TSSR successfully achieves parallel generation of high-fidelity 3D meshes through strategic decoupling of topology and shape generation, enhanced architectural design, and topological constraints, representing a significant advancement in 3D mesh generation.

Abstract: In this paper, we introduce Topology Sculptor, Shape Refiner (TSSR), a novel method for generating high-quality, artist-style 3D meshes based on Discrete Diffusion Models (DDMs). Our primary motivation for TSSR is to achieve highly accurate token prediction while enabling parallel generation, a significant advantage over sequential autoregressive methods. By allowing TSSR to “see” all mesh tokens concurrently, we unlock a new level of efficiency and control. We leverage this parallel generation capability through three key innovations: 1) Decoupled Training and Hybrid Inference, which distinctly separates the DDM-based generation into a topology sculpting stage and a subsequent shape refinement stage. This strategic decoupling enables TSSR to effectively capture both intricate local topology and overarching global shape. 2) An Improved Hourglass Architecture, featuring bidirectional attention enriched by face-vertex-sequence level Rotational Positional Embeddings (RoPE), thereby capturing richer contextual information across the mesh structure. 3) A novel Connection Loss, which acts as a topological constraint to further enhance the realism and fidelity of the generated meshes. Extensive experiments on complex datasets demonstrate that TSSR generates high-quality 3D artist-style meshes, capable of achieving up to 10,000 faces at a remarkable spatial resolution of $1024^3$. The code will be released at: https://github.com/psky1111/Tencent-TSSR.

Method: Proposes two components: (1) Object-Centric Semantic Grounding for object-level annotations in 3DGS, and (2) Physics-Aware Execution Jointing that embeds collision objects and constructs physical interfaces. Also releases InteriorGS dataset and SAGE-Bench benchmark.

Result: 3DGS scene data is more difficult to converge but exhibits strong generalizability, improving baseline performance by 31% on VLN-CE Unseen task. Created 1K object-annotated 3DGS indoor scenes and 2M VLN data benchmark.

Conclusion: SAGE-3D successfully bridges the sim-to-real gap by making 3DGS semantically and physically aligned for VLN, demonstrating improved performance and generalizability while providing new datasets and benchmarks.

Abstract: 3D Gaussian Splatting (3DGS), a 3D representation method with photorealistic real-time rendering capabilities, is regarded as an effective tool for narrowing the sim-to-real gap. However, it lacks fine-grained semantics and physical executability for Visual-Language Navigation (VLN). To address this, we propose SAGE-3D (Semantically and Physically Aligned Gaussian Environments for 3D Navigation), a new paradigm that upgrades 3DGS into an executable, semantically and physically aligned environment. It comprises two components: (1) Object-Centric Semantic Grounding, which adds object-level fine-grained annotations to 3DGS; and (2) Physics-Aware Execution Jointing, which embeds collision objects into 3DGS and constructs rich physical interfaces. We release InteriorGS, containing 1K object-annotated 3DGS indoor scene data, and introduce SAGE-Bench, the first 3DGS-based VLN benchmark with 2M VLN data. Experiments show that 3DGS scene data is more difficult to converge, while exhibiting strong generalizability, improving baseline performance by 31% on the VLN-CE Unseen task. The data and code will be available soon.

Method: Two-stage coarse-to-fine pipeline: Global Semantic Exploration (GSE) for instruction-guided reasoning and coarse target region generation, and Localized Perceptual Refinement (LPR) for accurate bounding box and segmentation mask refinement. Uses locate-informed retrospective reward to couple stages.

Result: Outperforms state-of-the-art MLLM-based approaches on both instruction-guided segmentation and visual reasoning tasks on FineRS-4k and public datasets.

Conclusion: FineRS effectively addresses the challenge of localizing extremely small objects in high-resolution scenes through its two-stage reinforcement learning framework and novel dataset.

Abstract: Multi-modal Large Language Models (MLLMs) have shown remarkable capabilities across a wide range of vision-language tasks. However, due to the restricted input resolutions, MLLMs face significant challenges in precisely understanding and localizing visual details in high-resolution images – particularly when dealing with extra-small objects embedded in cluttered contexts. To address this issue, we propose \textsc{FineRS}, a two-stage MLLM-based reinforcement learning framework for jointly reasoning and segmenting extremely small objects within high-resolution scenes. \textsc{FineRS} adopts a coarse-to-fine pipeline comprising Global Semantic Exploration (GSE) and Localized Perceptual Refinement (LPR). Specifically, GSE performs instruction-guided reasoning to generate a textural response and a coarse target region, while LPR refines this region to produce an accurate bounding box and segmentation mask. To couple the two stages, we introduce a locate-informed retrospective reward, where LPR’s outputs are used to optimize GSE for more robust coarse region exploration. % Additionally, we present \textsc{FineRS}-4k, a new dataset for evaluating MLLMs on attribute-level reasoning and pixel-level segmentation on subtle, small-scale targets in complex high-resolution scenes. Experimental results on \textsc{FineRS}-4k and public datasets demonstrate that our method consistently outperforms state-of-the-art MLLM-based approaches on both instruction-guided segmentation and visual reasoning tasks.

Method: VL-SAE uses a sparse autoencoder with distance-based encoder and modality-specific decoders, encouraging semantically similar representations to exhibit consistent neuron activations through self-supervised training based on cosine similarity.

Result: Experiments across VLMs (CLIP, LLaVA) show VL-SAE’s superior capability in interpreting and enhancing vision-language alignment, improving performance in zero-shot image classification and hallucination elimination.

Conclusion: VL-SAE successfully interprets vision-language alignment by mapping representations to concepts and enhances alignment at the concept level, benefiting downstream tasks.

Abstract: The alignment of vision-language representations endows current Vision-Language Models (VLMs) with strong multi-modal reasoning capabilities. However, the interpretability of the alignment component remains uninvestigated due to the difficulty in mapping the semantics of multi-modal representations into a unified concept set. To address this problem, we propose VL-SAE, a sparse autoencoder that encodes vision-language representations into its hidden activations. Each neuron in its hidden layer correlates to a concept represented by semantically similar images and texts, thereby interpreting these representations with a unified concept set. To establish the neuron-concept correlation, we encourage semantically similar representations to exhibit consistent neuron activations during self-supervised training. First, to measure the semantic similarity of multi-modal representations, we perform their alignment in an explicit form based on cosine similarity. Second, we construct the VL-SAE with a distance-based encoder and two modality-specific decoders to ensure the activation consistency of semantically similar representations. Experiments across multiple VLMs (e.g., CLIP, LLaVA) demonstrate the superior capability of VL-SAE in interpreting and enhancing the vision-language alignment. For interpretation, the alignment between vision and language representations can be understood by comparing their semantics with concepts. For enhancement, the alignment can be strengthened by aligning vision-language representations at the concept level, contributing to performance improvements in downstream tasks, including zero-shot image classification and hallucination elimination. Codes are available at https://github.com/ssfgunner/VL-SAE.

Method: FORM consists of two components: a morphology encoder trained via multi-channel VQGAN to learn 3D cell shape representations, and a diffusion-based perturbation trajectory module that models morphology evolution across perturbation conditions. It was trained on 65,000+ 3D cell volumes.

Result: FORM supports unconditional morphology synthesis and conditional simulation of perturbed cell states. It can predict downstream signaling activity, simulate combinatorial perturbation effects, and model morphodynamic transitions between states of unseen perturbations.

Conclusion: FORM and the accompanying MorphoEval benchmarking suite advance toward realizing the 3D virtual cell by linking morphology, perturbation, and function through high-resolution predictive simulation.

Abstract: Understanding how cells respond to external stimuli is a central challenge in biomedical research and drug development. Current computational frameworks for modelling cellular responses remain restricted to two-dimensional representations, limiting their capacity to capture the complexity of cell morphology under perturbation. This dimensional constraint poses a critical bottleneck for the development of accurate virtual cell models. Here, we present FORM, a machine learning framework for predicting perturbation-induced changes in three-dimensional cellular structure. FORM consists of two components: a morphology encoder, trained end-to-end via a novel multi-channel VQGAN to learn compact 3D representations of cell shape, and a diffusion-based perturbation trajectory module that captures how morphology evolves across perturbation conditions. Trained on a large-scale dataset of over 65,000 multi-fluorescence 3D cell volumes spanning diverse chemical and genetic perturbations, FORM supports both unconditional morphology synthesis and conditional simulation of perturbed cell states. Beyond generation, FORM can predict downstream signalling activity, simulate combinatorial perturbation effects, and model morphodynamic transitions between states of unseen perturbations. To evaluate performance, we introduce MorphoEval, a benchmarking suite that quantifies perturbation-induced morphological changes in structural, statistical, and biological dimensions. Together, FORM and MorphoEval work toward the realisation of the 3D virtual cell by linking morphology, perturbation, and function through high-resolution predictive simulation.

Method: Proposes CNN-Transformer-CLIP (CT-CLIP) framework with CNN for local lesion details, Vision Transformer for global structural relationships, Adaptive Feature Fusion Module for dynamic fusion, and multimodal image-text learning using pre-trained CLIP weights.

Result: Achieves 97.38% accuracy on public apple disease dataset and 96.12% on self-built dataset, outperforming baseline methods and showing strong performance under few-shot conditions.

Conclusion: CT-CLIP demonstrates strong capabilities in agricultural disease recognition, enhances accuracy under complex environmental conditions, and provides an innovative solution for automated disease recognition in agricultural applications.

Abstract: In complex orchard environments, the phenotypic heterogeneity of different apple leaf diseases, characterized by significant variation among lesions, poses a challenge to traditional multi-scale feature fusion methods. These methods only integrate multi-layer features extracted by convolutional neural networks (CNNs) and fail to adequately account for the relationships between local and global features. Therefore, this study proposes a multi-branch recognition framework named CNN-Transformer-CLIP (CT-CLIP). The framework synergistically employs a CNN to extract local lesion detail features and a Vision Transformer to capture global structural relationships. An Adaptive Feature Fusion Module (AFFM) then dynamically fuses these features, achieving optimal coupling of local and global information and effectively addressing the diversity in lesion morphology and distribution. Additionally, to mitigate interference from complex backgrounds and significantly enhance recognition accuracy under few-shot conditions, this study proposes a multimodal image-text learning approach. By leveraging pre-trained CLIP weights, it achieves deep alignment between visual features and disease semantic descriptions. Experimental results show that CT-CLIP achieves accuracies of 97.38% and 96.12% on a publicly available apple disease and a self-built dataset, outperforming several baseline methods. The proposed CT-CLIP demonstrates strong capabilities in recognizing agricultural diseases, significantly enhances identification accuracy under complex environmental conditions, provides an innovative and practical solution for automated disease recognition in agricultural applications.

Method: Proposes a Dynamic Semantic Relevance Generator combined with Transformer correlation maps to explore semantic relevance, enhancing search region’s ability to extract important information from templates. Also includes a pruning method for speed optimization.

Result: Achieves competitive performance on multiple UAV tracking datasets, with multiple model variants providing flexible speed-accuracy trade-offs for different computational resources.

Conclusion: The proposed framework effectively addresses semantic awareness limitations in UAV tracking, improving accuracy and robustness while maintaining practical deployment flexibility through speed-accuracy trade-offs.

Abstract: UAV tracking can be widely applied in scenarios such as disaster rescue, environmental monitoring, and logistics transportation. However, existing UAV tracking methods predominantly emphasize speed and lack exploration in semantic awareness, which hinders the search region from extracting accurate localization information from the template. The limitation results in suboptimal performance under typical UAV tracking challenges such as camera motion, fast motion, and low resolution, etc. To address this issue, we propose a dynamic semantic aware correlation modeling tracking framework. The core of our framework is a Dynamic Semantic Relevance Generator, which, in combination with the correlation map from the Transformer, explore semantic relevance. The approach enhances the search region’s ability to extract important information from the template, improving accuracy and robustness under the aforementioned challenges. Additionally, to enhance the tracking speed, we design a pruning method for the proposed framework. Therefore, we present multiple model variants that achieve trade-offs between speed and accuracy, enabling flexible deployment according to the available computational resources. Experimental results validate the effectiveness of our method, achieving competitive performance on multiple UAV tracking datasets. The code is available at https://github.com/zxyyxzz/DSATrack.

Method: Introduces gaze-regularized attention mechanism that aligns model focus with human visual gaze during training, without requiring gaze at inference.

Result: Improves semantic prediction scores by up to 11% for future event prediction and around 7% for current activity understanding compared to baseline models.

Conclusion: Establishes foundation for using human gaze to enhance VLM predictive capabilities in real-world scenarios like assistive robots and human-machine collaboration.

Abstract: Eye gaze offers valuable cues about attention, short-term intent, and future actions, making it a powerful signal for modeling egocentric behavior. In this work, we propose a gaze-regularized framework that enhances VLMs for two key egocentric understanding tasks: fine-grained future event prediction and current activity understanding. Unlike prior approaches that rely solely on visual inputs or use gaze as an auxiliary input signal , our method uses gaze only during training. We introduce a gaze-regularized attention mechanism that aligns model focus with human visual gaze. This design is flexible and modular, allowing it to generalize across multiple VLM architectures that utilize attention. Experimental results show that our approach improves semantic prediction scores by up to 11 for future event prediction and around 7 for current activity understanding, compared to the corresponding baseline models trained without gaze regularization. These results highlight the value of gaze-guided training in improving the accuracy and robustness of egocentric VLMs. Overall, this work establishes a foundation for using human gaze to enhance the predictive capabilities of VLMs in real-world scenarios like assistive robots and human-machine collaboration. Code and additional information is available at: https://github.com/anupampani/Gaze-VLM

Method: 2.5D U-Net++ with ResNet-34 encoder trained jointly across anatomical regions with fine-tuning. Used patch-based normalized inputs with L1 + IMPACT-Synth loss (combining SAM and TotalSegmentator). Compared Elastix (mutual information) vs IMPACT (feature-based) registration strategies.

Result: IMPACT-based registration achieved more accurate anatomical alignments and lower MAE on local tests, while Elastix-aligned data scored higher on public validation due to registration bias in evaluation pipeline.

Conclusion: Registration errors propagate into supervised learning, influencing both training and evaluation. IMPACT registration mitigates this bias and supports development of more robust sCT synthesis models with better anatomical fidelity.

Abstract: We participated in the SynthRAD2025 challenge (Tasks 1 and 2) with a unified pipeline for synthetic CT (sCT) generation from MRI and CBCT, implemented using the KonfAI framework. Our model is a 2.5D U-Net++ with a ResNet-34 encoder, trained jointly across anatomical regions and fine-tuned per region. The loss function combined pixel-wise L1 loss with IMPACT-Synth, a perceptual loss derived from SAM and TotalSegmentator to enhance structural fidelity. Training was performed using AdamW (initial learning rate = 0.001, halved every 25k steps) on patch-based, normalized, body-masked inputs (320x320 for MRI, 256x256 for CBCT), with random flipping as the only augmentation. No post-processing was applied. Final predictions leveraged test-time augmentation and five-fold ensembling. The best model was selected based on validation MAE. Two registration strategies were evaluated: (i) Elastix with mutual information, consistent with the challenge pipeline, and (ii) IMPACT, a feature-based similarity metric leveraging pretrained segmentation networks. On the local test sets, IMPACT-based registration achieved more accurate and anatomically consistent alignments than mutual-information-based registration, resulting in improved sCT synthesis with lower MAE and more realistic anatomical structures. On the public validation set, however, models trained with Elastix-aligned data achieved higher scores, reflecting a registration bias favoring alignment strategies consistent with the evaluation pipeline. This highlights how registration errors can propagate into supervised learning, influencing both training and evaluation, and potentially inflating performance metrics at the expense of anatomical fidelity. By promoting anatomically consistent alignment, IMPACT helps mitigate this bias and supports the development of more robust and generalizable sCT synthesis models.

Method: Joint end-to-end training strategy where diffusion model is conditioned on target quality level from available bandwidth, using lightweight quality embedding to guide denoising trajectory and enable adaptive early-stop sampling.

Result: Significantly improves visual fidelity of bandwidth-adapted generations compared to naive early-stopping, with minimal architectural changes required.

Conclusion: Offers a promising solution for efficient image delivery in bandwidth-constrained environments by enabling quality-adaptive generation based on transmission conditions.

Abstract: In this work, we propose a novel framework to enable diffusion models to adapt their generation quality based on real-time network bandwidth constraints. Traditional diffusion models produce high-fidelity images by performing a fixed number of denoising steps, regardless of downstream transmission limitations. However, in practical cloud-to-device scenarios, limited bandwidth often necessitates heavy compression, leading to loss of fine textures and wasted computation. To address this, we introduce a joint end-to-end training strategy where the diffusion model is conditioned on a target quality level derived from the available bandwidth. During training, the model learns to adaptively modulate the denoising process, enabling early-stop sampling that maintains perceptual quality appropriate to the target transmission condition. Our method requires minimal architectural changes and leverages a lightweight quality embedding to guide the denoising trajectory. Experimental results demonstrate that our approach significantly improves the visual fidelity of bandwidth-adapted generations compared to naive early-stopping, offering a promising solution for efficient image delivery in bandwidth-constrained environments. Code is available at: https://github.com/xzhang9308/BADiff.

Method: Proposes TerraGen with a geographic-spatial layout encoder that unifies bounding box and segmentation mask inputs, uses multi-scale injection scheme and mask-weighted loss to encode spatial constraints, and creates a large-scale multi-task dataset with 45k images.

Result: TerraGen achieves the best generation image quality across diverse tasks and serves as an effective universal data-augmentation generator, significantly enhancing downstream task performance with robust cross-task generalization in both full-data and few-shot scenarios.

Conclusion: TerraGen provides a unified solution for spatially controllable remote sensing image generation that overcomes task isolation limitations and demonstrates strong performance across multiple vision tasks.

Abstract: Remote sensing vision tasks require extensive labeled data across multiple, interconnected domains. However, current generative data augmentation frameworks are task-isolated, i.e., each vision task requires training an independent generative model, and ignores the modeling of geographical information and spatial constraints. To address these issues, we propose \textbf{TerraGen}, a unified layout-to-image generation framework that enables flexible, spatially controllable synthesis of remote sensing imagery for various high-level vision tasks, e.g., detection, segmentation, and extraction. Specifically, TerraGen introduces a geographic-spatial layout encoder that unifies bounding box and segmentation mask inputs, combined with a multi-scale injection scheme and mask-weighted loss to explicitly encode spatial constraints, from global structures to fine details. Also, we construct the first large-scale multi-task remote sensing layout generation dataset containing 45k images and establish a standardized evaluation protocol for this task. Experimental results show that our TerraGen can achieve the best generation image quality across diverse tasks. Additionally, TerraGen can be used as a universal data-augmentation generator, enhancing downstream task performance significantly and demonstrating robust cross-task generalisation in both full-data and few-shot scenarios.

Method: Multi-stage mechanism with global depth regularization for alignment, graph-based optimal seam computation with soft-seam diffusion, and reparameterization for efficiency optimization.

Result: The method effectively mitigates parallax-induced alignment errors and achieves natural, seamless stitching results with improved computational efficiency.

Conclusion: The proposed depth-consistency-constrained approach demonstrates superior performance over existing methods in handling parallax challenges in image stitching.

Abstract: Image stitching synthesizes images captured from multiple perspectives into a single image with a broader field of view. The significant variations in object depth often lead to large parallax, resulting in ghosting and misalignment in the stitched results. To address this, we propose a depth-consistency-constrained seamless-free image stitching method. First, to tackle the multi-view alignment difficulties caused by parallax, a multi-stage mechanism combined with global depth regularization constraints is developed to enhance the alignment accuracy of the same apparent target across different depth ranges. Second, during the multi-view image fusion process, an optimal stitching seam is determined through graph-based low-cost computation, and a soft-seam region is diffused to precisely locate transition areas, thereby effectively mitigating alignment errors induced by parallax and achieving natural and seamless stitching results. Furthermore, considering the computational overhead in the shift regression process, a reparameterization strategy is incorporated to optimize the structural design, significantly improving algorithm efficiency while maintaining optimal performance. Extensive experiments demonstrate the superior performance of the proposed method against the existing methods. Code is available at https://github.com/DLUT-YRH/DSFN.

Method: Proposes MUVR benchmark with three versions (Base, Filter, QA) containing 53K untrimmed videos from Bilibili, 1,050 multi-modal queries, and 84K matches. Features multi-level visual correspondence across six levels and supports various query types including long text descriptions, video tag prompts, and mask prompts.

Result: Extensive evaluations of 3 state-of-the-art video retrieval models, 6 image-based VLMs, and 10 MLLMs revealed limitations in processing untrimmed videos and multi-modal queries, as well as deficiencies in MLLMs’ multi-video understanding and reranking abilities.

Conclusion: MUVR provides a comprehensive benchmark that exposes current limitations in video retrieval methods and MLLMs, offering a foundation for future improvements in untrimmed video retrieval and multi-modal query processing.

Abstract: We propose the Multi-modal Untrimmed Video Retrieval task, along with a new benchmark (MUVR) to advance video retrieval for long-video platforms. MUVR aims to retrieve untrimmed videos containing relevant segments using multi-modal queries. It has the following features: 1) Practical retrieval paradigm: MUVR supports video-centric multi-modal queries, expressing fine-grained retrieval needs through long text descriptions, video tag prompts, and mask prompts. It adopts a one-to-many retrieval paradigm and focuses on untrimmed videos, tailored for long-video platform applications. 2) Multi-level visual correspondence: To cover common video categories (e.g., news, travel, dance) and precisely define retrieval matching criteria, we construct multi-level visual correspondence based on core video content (e.g., news events, travel locations, dance moves) which users are interested in and want to retrieve. It covers six levels: copy, event, scene, instance, action, and others. 3) Comprehensive evaluation criteria: We develop 3 versions of MUVR (i.e., Base, Filter, QA). MUVR-Base/Filter evaluates retrieval models, while MUVR-QA assesses MLLMs in a question-answering format. We also propose a Reranking Score to evaluate the reranking ability of MLLMs. MUVR consists of 53K untrimmed videos from the video platform Bilibili, with 1,050 multi-modal queries and 84K matches. Extensive evaluations of 3 state-of-the-art video retrieval models, 6 image-based VLMs, and 10 MLLMs are conducted. MUVR reveals the limitations of retrieval methods in processing untrimmed videos and multi-modal queries, as well as MLLMs in multi-video understanding and reranking. Our code and benchmark is available at https://github.com/debby-0527/MUVR.

Method: Uses pretrained vision-language model with universal prompting to discover diverse concept axes, and a universal concept encoder with compositional anchoring objective to independently manipulate axes without parameter overhead.

Result: Demonstrated superior editing capabilities on ImageNet, CelebA-HQ, and AFHQ datasets, handling diverse real-world concepts that cannot be manually predefined, with strong compositional generalization.

Conclusion: The framework enables scalable, adaptive visual concept learning in real-world scenes, outperforming existing methods in concept editing and generalization without requiring predefined axes or additional parameters per concept.

Abstract: Human intelligence effortlessly interprets visual scenes along a rich spectrum of semantic dimensions. However, existing approaches to language-grounded visual concept learning are limited to a few predefined primitive axes, such as color and shape, and are typically explored in synthetic datasets. In this work, we propose a scalable framework that adaptively identifies image-related concept axes and grounds visual concepts along these axes in real-world scenes. Leveraging a pretrained vision-language model and our universal prompting strategy, our framework identifies a diverse image-related axes without any prior knowledge. Our universal concept encoder adaptively binds visual features to the discovered axes without introducing additional model parameters for each concept. To ground visual concepts along the discovered axes, we optimize a compositional anchoring objective, which ensures that each axis can be independently manipulated without affecting others. We demonstrate the effectiveness of our framework on subsets of ImageNet, CelebA-HQ, and AFHQ, showcasing superior editing capabilities across diverse real-world concepts that are too varied to be manually predefined. Our method also exhibits strong compositional generalization, outperforming existing visual concept learning and text-based editing methods. The code is available at https://github.com/whieya/Language-grounded-VCL.

Method: Jointly models part geometry and articulation dynamics using sparse voxel representations embedded in a unified latent space via VAE, trains latent diffusion model for sampling, and uses articulation-aware Gaussian decoder for photorealistic reconstruction.

Result: Outperforms existing approaches on furniture-like objects from PartNet-Mobility and ACD datasets in geometric consistency and appearance fidelity, handling articulation-dependent visibility changes effectively.

Conclusion: ArtiLatent provides a scalable solution for articulated 3D object synthesis and manipulation, enabling realistic generation of articulated objects with proper geometry and appearance across different articulation states.

Abstract: We propose ArtiLatent, a generative framework that synthesizes human-made 3D objects with fine-grained geometry, accurate articulation, and realistic appearance. Our approach jointly models part geometry and articulation dynamics by embedding sparse voxel representations and associated articulation properties, including joint type, axis, origin, range, and part category, into a unified latent space via a variational autoencoder. A latent diffusion model is then trained over this space to enable diverse yet physically plausible sampling. To reconstruct photorealistic 3D shapes, we introduce an articulation-aware Gaussian decoder that accounts for articulation-dependent visibility changes (e.g., revealing the interior of a drawer when opened). By conditioning appearance decoding on articulation state, our method assigns plausible texture features to regions that are typically occluded in static poses, significantly improving visual realism across articulation configurations. Extensive experiments on furniture-like objects from PartNet-Mobility and ACD datasets demonstrate that ArtiLatent outperforms existing approaches in geometric consistency and appearance fidelity. Our framework provides a scalable solution for articulated 3D object synthesis and manipulation.

Method: OpenHype represents scene hierarchies using continuous hyperbolic latent space, leveraging hyperbolic geometry properties to naturally encode multi-scale relationships and enable smooth hierarchy traversal through geodesic paths.

Result: The method outperforms state-of-the-art approaches on standard benchmarks, demonstrating superior efficiency and adaptability in 3D scene understanding.

Conclusion: OpenHype provides an effective solution for hierarchical 3D scene representation that overcomes limitations of existing methods through continuous hyperbolic latent space modeling.

Abstract: Modeling the inherent hierarchical structure of 3D objects and 3D scenes is highly desirable, as it enables a more holistic understanding of environments for autonomous agents. Accomplishing this with implicit representations, such as Neural Radiance Fields, remains an unexplored challenge. Existing methods that explicitly model hierarchical structures often face significant limitations: they either require multiple rendering passes to capture embeddings at different levels of granularity, significantly increasing inference time, or rely on predefined, closed-set discrete hierarchies that generalize poorly to the diverse and nuanced structures encountered by agents in the real world. To address these challenges, we propose OpenHype, a novel approach that represents scene hierarchies using a continuous hyperbolic latent space. By leveraging the properties of hyperbolic geometry, OpenHype naturally encodes multi-scale relationships and enables smooth traversal of hierarchies through geodesic paths in latent space. Our method outperforms state-of-the-art approaches on standard benchmarks, demonstrating superior efficiency and adaptability in 3D scene understanding.

Method: PhysWorld constructs physics-consistent digital twins in an MPM simulator through constitutive model selection and global-to-local optimization. It then applies part-aware perturbations to generate diverse motion patterns, and trains a lightweight GNN-based world model with embedded physical properties.

Result: PhysWorld achieves accurate and fast future predictions for various deformable objects and generalizes well to novel interactions. It enables inference speeds 47 times faster than the state-of-the-art PhysTwin method while maintaining competitive performance.

Conclusion: The proposed framework successfully overcomes data scarcity by using simulator-generated demonstrations to train efficient world models, enabling fast and accurate physics predictions for deformable objects with real-world applicability.

Abstract: Interactive world models that simulate object dynamics are crucial for robotics, VR, and AR. However, it remains a significant challenge to learn physics-consistent dynamics models from limited real-world video data, especially for deformable objects with spatially-varying physical properties. To overcome the challenge of data scarcity, we propose PhysWorld, a novel framework that utilizes a simulator to synthesize physically plausible and diverse demonstrations to learn efficient world models. Specifically, we first construct a physics-consistent digital twin within MPM simulator via constitutive model selection and global-to-local optimization of physical properties. Subsequently, we apply part-aware perturbations to the physical properties and generate various motion patterns for the digital twin, synthesizing extensive and diverse demonstrations. Finally, using these demonstrations, we train a lightweight GNN-based world model that is embedded with physical properties. The real video can be used to further refine the physical properties. PhysWorld achieves accurate and fast future predictions for various deformable objects, and also generalizes well to novel interactions. Experiments show that PhysWorld has competitive performance while enabling inference speeds 47 times faster than the recent state-of-the-art method, i.e., PhysTwin.

Method: Uses streaming input to pre-trained VLMs with LSTM-inspired prediction mechanism to model temporal dependencies, plus a scoring queue and anomaly prior to dynamically store recent scores and guide LLMs in distinguishing normal vs abnormal behaviors.

Result: Outperforms state-of-the-art methods on UCF-Crime and XD-Violence datasets and is competitive with weakly supervised methods without requiring any training.

Conclusion: MoniTor effectively addresses online VAD challenges by leveraging pre-trained models with temporal modeling and dynamic scoring mechanisms, achieving strong performance without training.

Abstract: Video Anomaly Detection (VAD) aims to locate unusual activities or behaviors within videos. Recently, offline VAD has garnered substantial research attention, which has been invigorated by the progress in large language models (LLMs) and vision-language models (VLMs), offering the potential for a more nuanced understanding of anomalies. However, online VAD has seldom received attention due to real-time constraints and computational intensity. In this paper, we introduce a novel Memory-based online scoring queue scheme for Training-free VAD (MoniTor), to address the inherent complexities in online VAD. Specifically, MoniTor applies a streaming input to VLMs, leveraging the capabilities of pre-trained large-scale models. To capture temporal dependencies more effectively, we incorporate a novel prediction mechanism inspired by Long Short-Term Memory (LSTM) networks. This ensures the model can effectively model past states and leverage previous predictions to identify anomalous behaviors. Thereby, it better understands the current frame. Moreover, we design a scoring queue and an anomaly prior to dynamically store recent scores and cover all anomalies in the monitoring scenario, providing guidance for LLMs to distinguish between normal and abnormal behaviors over time. We evaluate MoniTor on two large datasets (i.e., UCF-Crime and XD-Violence) containing various surveillance and real-world scenarios. The results demonstrate that MoniTor outperforms state-of-the-art methods and is competitive with weakly supervised methods without training. Code is available at https://github.com/YsTvT/MoniTor.

Method: Proposes VidSplice framework with spaced-frame priors, CoSpliced Module for first-frame propagation strategy, and context controller module to constrain content distortion during generation.

Result: Extensive evaluations show competitive performance across diverse video inpainting scenarios with significant improvements in foreground alignment and motion stability.

Conclusion: VidSplice outperforms existing approaches by effectively addressing spatiotemporal stability issues in video inpainting through its novel decoupled framework design.

Abstract: Recent video inpainting methods often employ image-to-video (I2V) priors to model temporal consistency across masked frames. While effective in moderate cases, these methods struggle under severe content degradation and tend to overlook spatiotemporal stability, resulting in insufficient control over the latter parts of the video. To address these limitations, we decouple video inpainting into two sub-tasks: multi-frame consistent image inpainting and masked area motion propagation. We propose VidSplice, a novel framework that introduces spaced-frame priors to guide the inpainting process with spatiotemporal cues. To enhance spatial coherence, we design a CoSpliced Module to perform first-frame propagation strategy that diffuses the initial frame content into subsequent reference frames through a splicing mechanism. Additionally, we introduce a delicate context controller module that encodes coherent priors after frame duplication and injects the spliced video into the I2V generative backbone, effectively constraining content distortion during generation. Extensive evaluations demonstrate that VidSplice achieves competitive performance across diverse video inpainting scenarios. Moreover, its design significantly improves both foreground alignment and motion stability, outperforming existing approaches.

Method: Trains transcoder-based sparse autoencoders on BiomedCLIP diagnostic classifier to decompose medical image representations into interpretable visual patterns. Uses ensemble of 100 transcoders on multimodal embeddings from MIMIC-CXR dataset.

Result: Discovers ~5,000 monosemantic patterns spanning cardiac, pulmonary, pleural, structural, device, and artifact categories. Achieves competitive diagnostic accuracy on five key findings while providing transparent attribution through interpretable patterns with verifiable activation galleries.

Conclusion: Medical AI systems can be both accurate and interpretable. The key innovation is extracting interpretable features from classifiers trained on specific diagnostic objectives rather than general-purpose embeddings, supporting safer clinical deployment through transparent explanations.

Abstract: Deep learning models have achieved remarkable accuracy in chest X-ray diagnosis, yet their widespread clinical adoption remains limited by the black-box nature of their predictions. Clinicians require transparent, verifiable explanations to trust automated diagnoses and identify potential failure modes. We introduce CXR-LanIC (Language-Grounded Interpretable Classifier for Chest X-rays), a novel framework that addresses this interpretability challenge through task-aligned pattern discovery. Our approach trains transcoder-based sparse autoencoders on a BiomedCLIP diagnostic classifier to decompose medical image representations into interpretable visual patterns. By training an ensemble of 100 transcoders on multimodal embeddings from the MIMIC-CXR dataset, we discover approximately 5,000 monosemantic patterns spanning cardiac, pulmonary, pleural, structural, device, and artifact categories. Each pattern exhibits consistent activation behavior across images sharing specific radiological features, enabling transparent attribution where predictions decompose into 20-50 interpretable patterns with verifiable activation galleries. CXR-LanIC achieves competitive diagnostic accuracy on five key findings while providing the foundation for natural language explanations through planned large multimodal model annotation. Our key innovation lies in extracting interpretable features from a classifier trained on specific diagnostic objectives rather than general-purpose embeddings, ensuring discovered patterns are directly relevant to clinical decision-making, demonstrating that medical AI systems can be both accurate and interpretable, supporting safer clinical deployment through transparent, clinically grounded explanations.

Method: Reinterpret probing of intermediate activations through signal processing lens, analyze multiple samples systematically, rank attention heads based on relevance to target concepts, and selectively edit a small subset of heads.

Result: Found consistent patterns of head-level specialization across unimodal and multimodal transformers. Editing only 1% of heads (selected via the method) reliably suppresses or enhances targeted concepts in model outputs.

Conclusion: Attention layers contain interpretable and controllable structure, providing simple tools for understanding and editing large-scale generative models, validated across language tasks (QA, toxicity mitigation) and vision-language tasks (image classification, captioning).

Abstract: Language and vision-language models have shown impressive performance across a wide range of tasks, but their internal mechanisms remain only partly understood. In this work, we study how individual attention heads in text-generative models specialize in specific semantic or visual attributes. Building on an established interpretability method, we reinterpret the practice of probing intermediate activations with the final decoding layer through the lens of signal processing. This lets us analyze multiple samples in a principled way and rank attention heads based on their relevance to target concepts. Our results show consistent patterns of specialization at the head level across both unimodal and multimodal transformers. Remarkably, we find that editing as few as 1% of the heads, selected using our method, can reliably suppress or enhance targeted concepts in the model output. We validate our approach on language tasks such as question answering and toxicity mitigation, as well as vision-language tasks including image classification and captioning. Our findings highlight an interpretable and controllable structure within attention layers, offering simple tools for understanding and editing large-scale generative models.

Method: Dual-stream architecture with receptance-weighted key-value aggregation (recurrent transformer with linear complexity) and bidirectional tissue-cell interaction module for mutual attention between cellular cues and tissue environment.

Result: Outperforms existing models on four histopathological subtype classification benchmarks, demonstrating superior performance in fine-grained computational pathology.

Conclusion: Cell-level aggregation and tissue-cell interaction are critical components for accurate fine-grained analysis in computational pathology.

Abstract: Accurate interpretation of histopathological images demands integration of information across spatial and semantic scales, from nuclear morphology and cellular textures to global tissue organization and disease-specific patterns. Although recent foundation models in pathology have shown strong capabilities in capturing global tissue context, their omission of cell-level feature modeling remains a key limitation for fine-grained tasks such as cancer subtype classification. To address this, we propose a dual-stream architecture that models the interplay between macroscale tissue features and aggregated cellular representations. To efficiently aggregate information from large cell sets, we propose a receptance-weighted key-value aggregation model, a recurrent transformer that captures inter-cell dependencies with linear complexity. Furthermore, we introduce a bidirectional tissue-cell interaction module to enable mutual attention between localized cellular cues and their surrounding tissue environment. Experiments on four histopathological subtype classification benchmarks show that the proposed method outperforms existing models, demonstrating the critical role of cell-level aggregation and tissue-cell interaction in fine-grained computational pathology.

Method: Uses graph-weighted solution with online person identification label updates based on tracks and characteristic features. Includes deep investigation of MOT elements (feature extraction, tracking, community search) and incorporates person position estimation module.

Result: Tested on closed-area model recordings and public datasets of highly congested spaces, showing superiority over methods without position data. Achieved better performance than existing approaches.

Conclusion: GRAP-MOT demonstrates effective MOT in challenging occlusion scenarios. Analysis shows IDF1 metric is more adequate than MOTA for such comparisons. Code and dataset made publicly available.

Abstract: GRAP-MOT is a new approach for solving the person MOT problem dedicated to videos of closed areas with overlapping multi-camera views, where person occlusion frequently occurs. Our novel graph-weighted solution updates a person’s identification label online based on tracks and the person’s characteristic features. To find the best solution, we deeply investigated all elements of the MOT process, including feature extraction, tracking, and community search. Furthermore, GRAP-MOT is equipped with a person’s position estimation module, which gives additional key information to the MOT method, ensuring better results than methods without position data. We tested GRAP-MOT on recordings acquired in a closed-area model and on publicly available real datasets that fulfil the requirement of a highly congested space, showing the superiority of our proposition. Finally, we analyzed existing metrics used to compare MOT algorithms and concluded that IDF1 is more adequate than MOTA in such comparisons. We made our code, along with the acquired dataset, publicly available.

Method: Improved YOLOv11n model with wavelet convolution and coordinate convolution for frequency/spatial feature extraction, cascaded group attention mechanism to suppress ultrasound artifacts and occlusion interference, and enhanced bounding box localization.

Result: Achieved 78% detection accuracy, 2.5% improvement over YOLOv11n and 13.7% over YOLOv8. Superior performance in precision-recall curves, confidence scores, and occlusion scenarios with real-time processing capability.

Conclusion: EH-YOLOv11n provides a reliable computer-aided diagnosis solution for placental abruption with significant clinical application value, combining high accuracy with real-time processing.

Abstract: Placental abruption is a severe complication during pregnancy, and its early accurate diagnosis is crucial for ensuring maternal and fetal safety. Traditional ultrasound diagnostic methods heavily rely on physician experience, leading to issues such as subjective bias and diagnostic inconsistencies. This paper proposes an improved model, EH-YOLOv11n (Enhanced Hemorrhage-YOLOv11n), based on small-sample learning, aiming to achieve automatic detection of hematoma features in placental ultrasound images. The model enhances performance through multidimensional optimization: it integrates wavelet convolution and coordinate convolution to strengthen frequency and spatial feature extraction; incorporates a cascaded group attention mechanism to suppress ultrasound artifacts and occlusion interference, thereby improving bounding box localization accuracy. Experimental results demonstrate a detection accuracy of 78%, representing a 2.5% improvement over YOLOv11n and a 13.7% increase over YOLOv8. The model exhibits significant superiority in precision-recall curves, confidence scores, and occlusion scenarios. Combining high accuracy with real-time processing, this model provides a reliable solution for computer-aided diagnosis of placental abruption, holding significant clinical application value.

Method: Proposes GranViT with pretraining-adaptation framework and self-distillation mechanism. Uses Gran-29M dataset with 180M region-level annotations. Employs bounding-box-to-caption regression in pretraining and caption-to-bounding-box regression in adaptation, with self-distillation for localization constraints.

Result: GranViT surpasses existing vision encoders and attains strong transferability to varying LLMs. Achieves state-of-the-art results on fine-grained recognition, multimodal VQA, and OCR understanding.

Conclusion: The proposed GranViT successfully addresses the limitations of existing vision encoders by integrating fine-grained feature extraction with semantic alignment to LLMs, demonstrating superior performance across multiple vision-language tasks.

Abstract: Vision encoders are indispensable for allowing impressive performance of Multi-modal Large Language Models (MLLMs) in vision language tasks such as visual question answering and reasoning. However, existing vision encoders focus on global image representations but overlook fine-grained regional analysis. They are limited in fine grained perception due to the scarcity of fine grained annotated data and the lack of a fine grained pre-training paradigm. In this paper, we propose GranViT, a novel Vision Transformer that integrates fine-grained feature extraction with semantic alignment to Large Language Models (LLMs) via region level autoregressive training. We first construct Gran-29M, a dataset comprising 2million natural and OCR images paired with over 180 million high-quality region-level annotations, to enable large scale fine grained pretraining. Consequently, we develop a pretraining-adaptation framework along with a self distillation mechanism to train fine-grained GranViT on Gran-29M. We sufficiently exploit the fine-grained annotations from Gran-29M to resort to bounding-box-to-caption regression to enhance localized visual representation of the vision encoder in the pretraining and caption-to-bounding-box regression to improve vision feature utilization and localization for LLM in the adaptation. We further incorporate a self distillation mechanism that imposes explicit localization constraints on the vision encoder to strengthen its regional reasoning capability. Extensive experiments show that GranViT surpasses existing vision encoders and attains strong transferability to varying LLMs. Remarkably, it achieves state-of-the-art results on fine-grained recognition, multimodal VQA, and OCR understanding.

Method: The authors propose a unified perspective reframing conditional guidance as fixed point iterations, seeking a golden path where latents produce consistent outputs. They introduce Foresight Guidance (FSG) that prioritizes solving longer-interval subproblems in early diffusion stages with increased iterations.

Result: Extensive experiments across diverse datasets and model architectures validate FSG’s superiority over state-of-the-art methods in both image quality and computational efficiency.

Conclusion: The work offers novel perspectives for conditional guidance and unlocks the potential of adaptive design in text-to-image diffusion models.

Abstract: Classifier-Free Guidance (CFG) is an essential component of text-to-image diffusion models, and understanding and advancing its operational mechanisms remains a central focus of research. Existing approaches stem from divergent theoretical interpretations, thereby limiting the design space and obscuring key design choices. To address this, we propose a unified perspective that reframes conditional guidance as fixed point iterations, seeking to identify a golden path where latents produce consistent outputs under both conditional and unconditional generation. We demonstrate that CFG and its variants constitute a special case of single-step short-interval iteration, which is theoretically proven to exhibit inefficiency. To this end, we introduce Foresight Guidance (FSG), which prioritizes solving longer-interval subproblems in early diffusion stages with increased iterations. Extensive experiments across diverse datasets and model architectures validate the superiority of FSG over state-of-the-art methods in both image quality and computational efficiency. Our work offers novel perspectives for conditional guidance and unlocks the potential of adaptive design.

Method: Proposes Chunk-GRPO that groups consecutive steps into coherent chunks to capture temporal dynamics and optimizes policies at chunk level, with optional weighted sampling.

Result: Extensive experiments show Chunk-GRPO achieves superior results in both preference alignment and image quality compared to step-level approaches.

Conclusion: Chunk-level optimization shows promise for GRPO-based methods in text-to-image generation, effectively addressing previous limitations.

Abstract: Group Relative Policy Optimization (GRPO) has shown strong potential for flow-matching-based text-to-image (T2I) generation, but it faces two key limitations: inaccurate advantage attribution, and the neglect of temporal dynamics of generation. In this work, we argue that shifting the optimization paradigm from the step level to the chunk level can effectively alleviate these issues. Building on this idea, we propose Chunk-GRPO, the first chunk-level GRPO-based approach for T2I generation. The insight is to group consecutive steps into coherent ‘chunk’s that capture the intrinsic temporal dynamics of flow matching, and to optimize policies at the chunk level. In addition, we introduce an optional weighted sampling strategy to further enhance performance. Extensive experiments show that ChunkGRPO achieves superior results in both preference alignment and image quality, highlighting the promise of chunk-level optimization for GRPO-based methods.

Method: MATrack uses three collaborative modules: Multiscale Hierarchy Blende for feature consistency between static/dynamic templates, Adaptive Key Token Gate for object identification in complex backgrounds, and Nighttime Template Calibrator for stable long-sequence tracking.

Result: On UAVDark135 benchmark, MATrack achieves 5.9% precision, 5.4% normalized precision, and 4.2% AUC improvements over SOTA methods while maintaining 81 FPS real-time processing. Real-world UAV platform tests validate reliability.

Conclusion: MATrack provides stable and effective nighttime UAV tracking for critical robotics applications like search and rescue and border patrol, significantly outperforming existing methods while maintaining real-time performance.

Abstract: Nighttime UAV tracking faces significant challenges in real-world robotics operations. Low-light conditions not only limit visual perception capabilities, but cluttered backgrounds and frequent viewpoint changes also cause existing trackers to drift or fail during deployment. To address these difficulties, researchers have proposed solutions based on low-light enhancement and domain adaptation. However, these methods still have notable shortcomings in actual UAV systems: low-light enhancement often introduces visual artifacts, domain adaptation methods are computationally expensive and existing lightweight designs struggle to fully leverage dynamic object information. Based on an in-depth analysis of these key issues, we propose MATrack-a multiscale adaptive system designed specifically for nighttime UAV tracking. MATrack tackles the main technical challenges of nighttime tracking through the collaborative work of three core modules: Multiscale Hierarchy Blende (MHB) enhances feature consistency between static and dynamic templates. Adaptive Key Token Gate accurately identifies object information within complex backgrounds. Nighttime Template Calibrator (NTC) ensures stable tracking performance over long sequences. Extensive experiments show that MATrack achieves a significant performance improvement. On the UAVDark135 benchmark, its precision, normalized precision and AUC surpass state-of-the-art (SOTA) methods by 5.9%, 5.4% and 4.2% respectively, while maintaining a real-time processing speed of 81 FPS. Further tests on a real-world UAV platform validate the system’s reliability, demonstrating that MATrack can provide stable and effective nighttime UAV tracking support for critical robotics applications such as nighttime search and rescue and border patrol.

Method: TIGER uses a “text-first, image-later” paradigm that explicitly decouples glyph restoration from image enhancement. It first reconstructs precise text structures and then uses them to guide subsequent full-image super-resolution through glyph-to-image guidance.

Result: TIGER achieves state-of-the-art performance, enhancing readability while preserving overall image quality. The method was evaluated using the new UltraZoom-ST dataset with extreme zoom (×14.29).

Conclusion: The proposed two-stage framework successfully breaks the trade-off between image quality and text readability in super-resolution, demonstrating superior performance through explicit text structure reconstruction followed by guided image enhancement.

Abstract: Current generative super-resolution methods show strong performance on natural images but distort text, creating a fundamental trade-off between image quality and textual readability. To address this, we introduce \textbf{TIGER} (\textbf{T}ext-\textbf{I}mage \textbf{G}uided sup\textbf{E}r-\textbf{R}esolution), a novel two-stage framework that breaks this trade-off through a \textit{“text-first, image-later”} paradigm. \textbf{TIGER} explicitly decouples glyph restoration from image enhancement: it first reconstructs precise text structures and then uses them to guide subsequent full-image super-resolution. This glyph-to-image guidance ensures both high fidelity and visual consistency. To support comprehensive training and evaluation, we also contribute the \textbf{UltraZoom-ST} (UltraZoom-Scene Text), the first scene text dataset with extreme zoom (\textbf{$\times$14.29}). Extensive experiments show that \textbf{TIGER} achieves \textbf{state-of-the-art} performance, enhancing readability while preserving overall image quality.

Method: Proposed two deep learning architectures: feature-based ANN and CNN, trained on 59 patients’ data. Used interactive machine learning to iteratively improve annotation quality. Evaluated against state-of-the-art model for classifying short time windows of signal.

Result: Both models outperformed state-of-the-art (F1-scores: CNN=0.46, ANN=0.44) on 10 holdout test patients. Interactive learning demonstrated model robustness to noisy annotations. Simple architectures proved effective on complex, imperfectly annotated data.

Conclusion: Deep learning models with simple architectures are robust for automated interictal spike detection. Interactive machine learning enables faster data annotation and provides efficient tools for clinical epilepsy evaluation.

Abstract: In drug-resistant epilepsy, presurgical evaluation of epilepsy can be considered. Magnetoencephalography (MEG) has been shown to be an effective exam to inform the localization of the epileptogenic zone through the localization of interictal epileptic spikes. Manual detection of these pathological biomarkers remains a fastidious and error-prone task due to the high dimensionality of MEG recordings, and interrater agreement has been reported to be only moderate. Current automated methods are unsuitable for clinical practice, either requiring extensively annotated data or lacking robustness on non-typical data. In this work, we demonstrate that deep learning models can be used for detecting interictal spikes in MEG recordings, even when only temporal and single-expert annotations are available, which represents real-world clinical practice. We propose two model architectures: a feature-based artificial neural network (ANN) and a convolutional neural network (CNN), trained on a database of 59 patients, and evaluated against a state-of-the-art model to classify short time windows of signal. In addition, we employ an interactive machine learning strategy to iteratively improve our data annotation quality using intermediary model outputs. Both proposed models outperform the state-of-the-art model (F1-scores: CNN=0.46, ANN=0.44) when tested on 10 holdout test patients. The interactive machine learning strategy demonstrates that our models are robust to noisy annotations. Overall, results highlight the robustness of models with simple architectures when analyzing complex and imperfectly annotated data. Our method of interactive machine learning offers great potential for faster data annotation, while our models represent useful and efficient tools for automated interictal spikes detection.

Method: Multi-modal diffusion pipeline generates S3OD dataset with 139,000 high-resolution images using diffusion and DINO-v3 features, with iterative framework prioritizing challenging categories. Streamlined multi-mask decoder handles ambiguity by predicting multiple valid interpretations.

Result: Models trained solely on synthetic data achieve 20-50% error reduction in cross-dataset generalization. Fine-tuned versions reach state-of-the-art performance across DIS and HR-SOD benchmarks.

Conclusion: Large-scale synthetic data generation combined with ambiguity-aware architecture dramatically improves generalization in salient object detection tasks.

Abstract: Salient object detection exemplifies data-bounded tasks where expensive pixel-precise annotations force separate model training for related subtasks like DIS and HR-SOD. We present a method that dramatically improves generalization through large-scale synthetic data generation and ambiguity-aware architecture. We introduce S3OD, a dataset of over 139,000 high-resolution images created through our multi-modal diffusion pipeline that extracts labels from diffusion and DINO-v3 features. The iterative generation framework prioritizes challenging categories based on model performance. We propose a streamlined multi-mask decoder that naturally handles the inherent ambiguity in salient object detection by predicting multiple valid interpretations. Models trained solely on synthetic data achieve 20-50% error reduction in cross-dataset generalization, while fine-tuned versions reach state-of-the-art performance across DIS and HR-SOD benchmarks.

Method: Proposes Modest-Align with two complementary strategies: Random Perturbation (introduces controlled noise to simulate uncertainty) and Embedding Smoothing (calibrates similarity distributions in embedding space).

Result: Outperforms state-of-the-art methods in retrieval tasks, achieving competitive results with over 100x less training data and 600x less GPU time than CLIP across multiple benchmark datasets.

Conclusion: Modest-Align offers a practical and scalable solution for cross-modal alignment in real-world, low-resource scenarios by reducing overconfidence and improving performance on noisy or weakly aligned samples.

Abstract: Cross-modal alignment aims to map heterogeneous modalities into a shared latent space, as exemplified by models like CLIP, which benefit from large-scale image-text pretraining for strong recognition capabilities. However, when operating in resource-constrained settings with limited or low-quality data, these models often suffer from overconfidence and degraded performance due to the prevalence of ambiguous or weakly correlated image-text pairs. Current contrastive learning approaches, which rely on single positive pairs, further exacerbate this issue by reinforcing overconfidence on uncertain samples. To address these challenges, we propose Modest-Align, a lightweight alignment framework designed for robustness and efficiency. Our approach leverages two complementary strategies – Random Perturbation, which introduces controlled noise to simulate uncertainty, and Embedding Smoothing, which calibrates similarity distributions in the embedding space. These mechanisms collectively reduce overconfidence and improve performance on noisy or weakly aligned samples. Extensive experiments across multiple benchmark datasets demonstrate that Modest-Align outperforms state-of-the-art methods in retrieval tasks, achieving competitive results with over 100x less training data and 600x less GPU time than CLIP. Our method offers a practical and scalable solution for cross-modal alignment in real-world, low-resource scenarios.

Method: Align diffusion models using pairwise epipolar geometry constraints via preference-based optimization, enforcing geometric principles without requiring end-to-end differentiability. Training on static scenes with dynamic cameras ensures quality measurements.

Result: Classical geometric constraints provide more stable optimization signals than modern learned metrics, enabling generation of spatially consistent videos without compromising visual quality.

Conclusion: Bridging data-driven deep learning with classical geometric computer vision presents a practical method for generating geometrically consistent videos.

Abstract: Video generation models have progressed tremendously through large latent diffusion transformers trained with rectified flow techniques. Yet these models still struggle with geometric inconsistencies, unstable motion, and visual artifacts that break the illusion of realistic 3D scenes. 3D-consistent video generation could significantly impact numerous downstream applications in generation and reconstruction tasks. We explore how epipolar geometry constraints improve modern video diffusion models. Despite massive training data, these models fail to capture fundamental geometric principles underlying visual content. We align diffusion models using pairwise epipolar geometry constraints via preference-based optimization, directly addressing unstable camera trajectories and geometric artifacts through mathematically principled geometric enforcement. Our approach efficiently enforces geometric principles without requiring end-to-end differentiability. Evaluation demonstrates that classical geometric constraints provide more stable optimization signals than modern learned metrics, which produce noisy targets that compromise alignment quality. Training on static scenes with dynamic cameras ensures high-quality measurements while the model generalizes effectively to diverse dynamic content. By bridging data-driven deep learning with classical geometric computer vision, we present a practical method for generating spatially consistent videos without compromising visual quality.

Method: Proposes DAP-MAE with a heterogeneous domain adapter that uses adaptation mode during pre-training and fusion mode during fine-tuning, plus a domain feature generator to guide feature adaptation to various downstream tasks.

Result: Achieves 95.18% accuracy in object classification on ScanObjectNN and 88.45% in facial expression recognition on Bosphorus with only one pre-training, performing well across four different point cloud analysis tasks.

Conclusion: DAP-MAE effectively addresses cross-domain point cloud data scarcity by adaptively integrating knowledge from different domains while maintaining strong performance across multiple downstream tasks.

Abstract: Compared to 2D data, the scale of point cloud data in different domains available for training, is quite limited. Researchers have been trying to combine these data of different domains for masked autoencoder (MAE) pre-training to leverage such a data scarcity issue. However, the prior knowledge learned from mixed domains may not align well with the downstream 3D point cloud analysis tasks, leading to degraded performance. To address such an issue, we propose the Domain-Adaptive Point Cloud Masked Autoencoder (DAP-MAE), an MAE pre-training method, to adaptively integrate the knowledge of cross-domain datasets for general point cloud analysis. In DAP-MAE, we design a heterogeneous domain adapter that utilizes an adaptation mode during pre-training, enabling the model to comprehensively learn information from point clouds across different domains, while employing a fusion mode in the fine-tuning to enhance point cloud features. Meanwhile, DAP-MAE incorporates a domain feature generator to guide the adaptation of point cloud features to various downstream tasks. With only one pre-training, DAP-MAE achieves excellent performance across four different point cloud analysis tasks, reaching 95.18% in object classification on ScanObjectNN and 88.45% in facial expression recognition on Bosphorus.

Method: Proposes a stage-aware distillation strategy using Gompertz curve to dynamically adjust distillation loss weights, incorporates Wasserstein distance for feature-level discrepancy measurement, and gradient matching to align backward propagation behaviors.

Result: Extensive experiments on CIFAR-10 and CIFAR-100 show Gompertz-CNN outperforms traditional distillation methods, achieving up to 8% accuracy gain on CIFAR-10 and 4% on CIFAR-100 across various teacher-student architectures.

Conclusion: The Gompertz-CNN framework effectively addresses limitations of traditional knowledge distillation by modeling the student’s learning progression, resulting in significant performance improvements.

Abstract: This paper introduces a novel dynamic knowledge distillation framework, Gompertz-CNN, which integrates the Gompertz growth model into the training process to address the limitations of traditional knowledge distillation. Conventional methods often fail to capture the evolving cognitive capacity of student models, leading to suboptimal knowledge transfer. To overcome this, we propose a stage-aware distillation strategy that dynamically adjusts the weight of distillation loss based on the Gompertz curve, reflecting the student’s learning progression: slow initial growth, rapid mid-phase improvement, and late-stage saturation. Our framework incorporates Wasserstein distance to measure feature-level discrepancies and gradient matching to align backward propagation behaviors between teacher and student models. These components are unified under a multi-loss objective, where the Gompertz curve modulates the influence of distillation losses over time. Extensive experiments on CIFAR-10 and CIFAR-100 using various teacher-student architectures (e.g., ResNet50 and MobileNet_v2) demonstrate that Gompertz-CNN consistently outperforms traditional distillation methods, achieving up to 8% and 4% accuracy gains on CIFAR-10 and CIFAR-100, respectively.

Method: Uses distances between sparse wearable sensors (both on-body and cross-person) from UWB ranging, fuses them with inertial observations in structured state-space models, and employs a novel two-step optimization for global trajectory tracking.

Result: Outperforms previous state-of-the-art methods in accuracy and robustness across synthetic and real-world data, demonstrating promise for IMU+UWB-based multi-human motion capture in the wild.

Conclusion: The approach successfully addresses the limitations of both vision-based and purely IMU-based tracking by combining IMU and UWB technologies, enabling robust multi-person motion capture with accurate global positioning.

Abstract: Tracking human full-body motion using sparse wearable inertial measurement units (IMUs) overcomes the limitations of occlusion and instrumentation of the environment inherent in vision-based approaches. However, purely IMU-based tracking compromises translation estimates and accurate relative positioning between individuals, as inertial cues are inherently self-referential and provide no direct spatial reference for others. In this paper, we present a novel approach for robustly estimating body poses and global translation for multiple individuals by leveraging the distances between sparse wearable sensors - both on each individual and across multiple individuals. Our method Group Inertial Poser estimates these absolute distances between pairs of sensors from ultra-wideband ranging (UWB) and fuses them with inertial observations as input into structured state-space models to integrate temporal motion patterns for precise 3D pose estimation. Our novel two-step optimization further leverages the estimated distances for accurately tracking people’s global trajectories through the world. We also introduce GIP-DB, the first IMU+UWB dataset for two-person tracking, which comprises 200 minutes of motion recordings from 14 participants. In our evaluation, Group Inertial Poser outperforms previous state-of-the-art methods in accuracy and robustness across synthetic and real-world data, showing the promise of IMU+UWB-based multi-human motion capture in the wild. Code, models, dataset: https://github.com/eth-siplab/GroupInertialPoser

Method: Built on PyTorch Wildlife model, experimented with various LTR loss functions and LTR-sensitive regularization, evaluated on NACTI dataset and domain-shifted Reduced-Bias Test set from ENA-Detection.

Result: Achieved 99.40% Top-1 accuracy on NACTI test data (vs 95.51% baseline), and 52.55% accuracy on Reduced-Bias Test set (vs 51.20% with WCE loss), showing improved generalization under distribution shift.

Conclusion: LTR-enhancing scheduler choices consistently improve performance in wildlife domain, particularly with state-of-the-art LTR losses, though limitations remain including catastrophic breakdown for ‘Tail’ classes under severe domain shift.

Abstract: As most ‘‘in the wild’’ data collections of the natural world, the North America Camera Trap Images (NACTI) dataset shows severe long-tailed class imbalance, noting that the largest ‘Head’ class alone covers >50% of the 3.7M images in the corpus. Building on the PyTorch Wildlife model, we present a systematic study of Long-Tail Recognition methodologies for species recognition on the NACTI dataset covering experiments on various LTR loss functions plus LTR-sensitive regularisation. Our best configuration achieves 99.40% Top-1 accuracy on our NACTI test data split, substantially improving over a 95.51% baseline using standard cross-entropy with Adam. This also improves on previously reported top performance in MLWIC2 at 96.8% albeit using partly unpublished (potentially different) partitioning, optimiser, and evaluation protocols. To evaluate domain shifts (e.g. night-time captures, occlusion, motion-blur) towards other datasets we construct a Reduced-Bias Test set from the ENA-Detection dataset where our experimentally optimised long-tail enhanced model achieves leading 52.55% accuracy (up from 51.20% with WCE loss), demonstrating stronger generalisation capabilities under distribution shift. We document the consistent improvements of LTR-enhancing scheduler choices in this NACTI wildlife domain, particularly when in tandem with state-of-the-art LTR losses. We finally discuss qualitative and quantitative shortcomings that LTR methods cannot sufficiently address, including catastrophic breakdown for ‘Tail’ classes under severe domain shift. For maximum reproducibility we publish all dataset splits, key code, and full network weights.

Method: Use spatial proximity as a similarity measure - nearby synapses in the same neuron are assumed more similar than randomly selected synapses from different cells. Applied to Drosophila EM data without predefined class numbers.

Result: Successfully separated synapses into classes without supervised training, providing a principled approach to identify synapse types that span the structural range.

Conclusion: This unsupervised method enables synapse classification without prior knowledge of synapse types, offering a more flexible approach for discovering and characterizing synapse diversity in neural circuits.

Abstract: Separating synapses into different classes based on their appearance in EM images has many applications in biology. Examples may include assigning a neurotransmitter to a particular class, or separating synapses whose strength can be modulated from those whose strength is fixed. Traditionally, this has been done in a supervised manner, giving the classification algorithm examples of the different classes. Here we instead separate synapses into classes based only on the observation that nearby synapses in the same neuron are likely more similar than synapses chosen randomly from different cells. We apply our methodology to data from {\it Drosophila}. Our approach has the advantage that the number of synapse types does not need to be known in advance. It may also provide a principled way to select ground-truth that spans the range of synapse structure.

Method: Used UNI and Virchow2 foundation models as feature extractors, aggregated patch-level embeddings into slide-level features using mean-aggregation, and trained multiple classifiers (logistic regression, gradient-boosted trees, random forest). Applied data augmentation and image normalization for robustness.

Result: Virchow2 features outperformed UNI across most classifiers, with logistic regression achieving highest accuracy (90%) for Virchow2, though the difference was not statistically significant. Mean-aggregation provided reliable slide-level feature representations.

Conclusion: Foundation models show strong potential for automated WSI classification, providing a scalable approach for dermatopathological diagnosis and paving the way for future advancements in slide-level representation learning.

Abstract: The rapid generation of whole-slide images (WSIs) in dermatopathology necessitates automated methods for efficient processing and accurate classification. This study evaluates the performance of two foundation models, UNI and Virchow2, as feature extractors for classifying WSIs into three diagnostic categories: melanocytic, basaloid, and squamous lesions. Patch-level embeddings were aggregated into slide-level features using a mean-aggregation strategy and subsequently used to train multiple machine learning classifiers, including logistic regression, gradient-boosted trees, and random forest models. Performance was assessed using precision, recall, true positive rate, false positive rate, and the area under the receiver operating characteristic curve (AUROC) on the test set. Results demonstrate that patch-level features extracted using Virchow2 outperformed those extracted via UNI across most slide-level classifiers, with logistic regression achieving the highest accuracy (90%) for Virchow2, though the difference was not statistically significant. The study also explored data augmentation techniques and image normalization to enhance model robustness and generalizability. The mean-aggregation approach provided reliable slide-level feature representations. All experimental results and metrics were tracked and visualized using WandB.ai, facilitating reproducibility and interpretability. This research highlights the potential of foundation models for automated WSI classification, providing a scalable and effective approach for dermatopathological diagnosis while paving the way for future advancements in slide-level representation learning.

Method: Three core components: (1) data curation pipeline for high-quality scene blocks, (2) 3D block inpainting for context-aware scene extension, and (3) coarse-to-fine generation strategy for global layout plausibility and local fidelity.

Result: Achieves state-of-the-art performance in geometry reconstruction on 3D-FRONT dataset and uniquely supports infinite scene generation with photorealistic and structurally consistent outputs.

Conclusion: WorldGrow demonstrates capability for constructing large-scale virtual environments and has potential for building future world models.

Abstract: We tackle the challenge of generating the infinitely extendable 3D world – large, continuous environments with coherent geometry and realistic appearance. Existing methods face key challenges: 2D-lifting approaches suffer from geometric and appearance inconsistencies across views, 3D implicit representations are hard to scale up, and current 3D foundation models are mostly object-centric, limiting their applicability to scene-level generation. Our key insight is leveraging strong generation priors from pre-trained 3D models for structured scene block generation. To this end, we propose WorldGrow, a hierarchical framework for unbounded 3D scene synthesis. Our method features three core components: (1) a data curation pipeline that extracts high-quality scene blocks for training, making the 3D structured latent representations suitable for scene generation; (2) a 3D block inpainting mechanism that enables context-aware scene extension; and (3) a coarse-to-fine generation strategy that ensures both global layout plausibility and local geometric/textural fidelity. Evaluated on the large-scale 3D-FRONT dataset, WorldGrow achieves SOTA performance in geometry reconstruction, while uniquely supporting infinite scene generation with photorealistic and structurally consistent outputs. These results highlight its capability for constructing large-scale virtual environments and potential for building future world models.

Method: Train Faster R-CNN on aerial imagery from Glacier Bay National Park to detect harbor seals, then generate explanations using gradient-based class activation mapping (HiResCAM, LayerCAM), LIME, and perturbation-based explanations.

Result: Explanations focus on seal torsos and contours rather than background, removal of seals reduces detection confidence, and analysis reveals systematic errors like confusion between seals and black ice/rocks.

Conclusion: Pairing object detection with post-hoc explainability enables moving beyond black-box predictions toward auditable, decision-supporting tools for conservation monitoring, with actionable next steps for model improvement.

Abstract: Computer vision can accelerate ecological research and conservation monitoring, yet adoption in ecology lags in part because of a lack of trust in black-box neural-network-based models. We seek to address this challenge by applying post-hoc explanations to provide evidence for predictions and document limitations that are important to field deployment. Using aerial imagery from Glacier Bay National Park, we train a Faster R-CNN to detect pinnipeds (harbor seals) and generate explanations via gradient-based class activation mapping (HiResCAM, LayerCAM), local interpretable model-agnostic explanations (LIME), and perturbation-based explanations. We assess explanations along three axes relevant to field use: (i) localization fidelity: whether high-attribution regions coincide with the animal rather than background context; (ii) faithfulness: whether deletion/insertion tests produce changes in detector confidence; and (iii) diagnostic utility: whether explanations reveal systematic failure modes. Explanations concentrate on seal torsos and contours rather than surrounding ice/rock, and removal of the seals reduces detection confidence, providing model-evidence for true positives. The analysis also uncovers recurrent error sources, including confusion between seals and black ice and rocks. We translate these findings into actionable next steps for model development, including more targeted data curation and augmentation. By pairing object detection with post-hoc explainability, we can move beyond “black-box” predictions toward auditable, decision-supporting tools for conservation monitoring.

Method: Systematic analysis of DiT’s attention mechanism reveals its ability to extract foreground masks and identify matching points. The method generates an identity video, caches intermediate variables, and injects them into new videos to ensure consistency.

Result: Experimental results show BachVid achieves robust consistency in generated videos without requiring additional training.

Conclusion: BachVid offers an efficient solution for consistent video generation without relying on reference images or additional training.

Abstract: Diffusion Transformers (DiTs) have recently driven significant progress in text-to-video (T2V) generation. However, generating multiple videos with consistent characters and backgrounds remains a significant challenge. Existing methods typically rely on reference images or extensive training, and often only address character consistency, leaving background consistency to image-to-video models. We introduce BachVid, the first training-free method that achieves consistent video generation without needing any reference images. Our approach is based on a systematic analysis of DiT’s attention mechanism and intermediate features, revealing its ability to extract foreground masks and identify matching points during the denoising process. Our method leverages this finding by first generating an identity video and caching the intermediate variables, and then inject these cached variables into corresponding positions in newly generated videos, ensuring both foreground and background consistency across multiple videos. Experimental results demonstrate that BachVid achieves robust consistency in generated videos without requiring additional training, offering a novel and efficient solution for consistent video generation without relying on reference images or additional training.

Method: Treat geometric problems as images and train standard visual diffusion models to transform Gaussian noise into images representing valid approximate solutions that match exact ones.

Result: Successfully applied to three hard geometric problems: Inscribed Square Problem, Steiner Tree Problem, and Simple Polygon Problem, showing the model can learn to transform noisy geometric structures into correct configurations.

Conclusion: Operating in image space provides a general framework for approximating notoriously hard geometric problems and opens doors to tackling a wider class of challenging geometric tasks.

Abstract: In this paper we show that visual diffusion models can serve as effective geometric solvers: they can directly reason about geometric problems by working in pixel space. We first demonstrate this on the Inscribed Square Problem, a long-standing problem in geometry that asks whether every Jordan curve contains four points forming a square. We then extend the approach to two other well-known hard geometric problems: the Steiner Tree Problem and the Simple Polygon Problem. Our method treats each problem instance as an image and trains a standard visual diffusion model that transforms Gaussian noise into an image representing a valid approximate solution that closely matches the exact one. The model learns to transform noisy geometric structures into correct configurations, effectively recasting geometric reasoning as image generation. Unlike prior work that necessitates specialized architectures and domain-specific adaptations when applying diffusion to parametric geometric representations, we employ a standard visual diffusion model that operates on the visual representation of the problem. This simplicity highlights a surprising bridge between generative modeling and geometric problem solving. Beyond the specific problems studied here, our results point toward a broader paradigm: operating in image space provides a general and practical framework for approximating notoriously hard problems, and opens the door to tackling a far wider class of challenging geometric tasks.

Method: A self-reflective agent that systematically generates and tests hypotheses about visual attributes, using iterative refinement based on experimental outcomes and self-evaluation protocols.

Result: The agent’s performance consistently improves with self-reflection, showing significant performance increase over non-reflective baselines on a benchmark of 130 models across 18 categories. It also identifies real-world dependencies in CLIP’s vision encoder and YOLOv8 object detector.

Conclusion: The self-reflective framework effectively detects visual attribute dependencies in vision models, demonstrating improved performance through iterative reflection and validation.

Abstract: When a vision model performs image recognition, which visual attributes drive its predictions? Detecting unintended reliance on specific visual features is critical for ensuring model robustness, preventing overfitting, and avoiding spurious correlations. We introduce an automated framework for detecting such dependencies in trained vision models. At the core of our method is a self-reflective agent that systematically generates and tests hypotheses about visual attributes that a model may rely on. This process is iterative: the agent refines its hypotheses based on experimental outcomes and uses a self-evaluation protocol to assess whether its findings accurately explain model behavior. When inconsistencies arise, the agent self-reflects over its findings and triggers a new cycle of experimentation. We evaluate our approach on a novel benchmark of 130 models designed to exhibit diverse visual attribute dependencies across 18 categories. Our results show that the agent’s performance consistently improves with self-reflection, with a significant performance increase over non-reflective baselines. We further demonstrate that the agent identifies real-world visual attribute dependencies in state-of-the-art models, including CLIP’s vision encoder and the YOLOv8 object detector.

Method: Created MME benchmark with 14 subtasks covering perception and cognition abilities. Used manually designed instruction-answer pairs to avoid data leakage from public datasets. Implemented concise instruction design for fair comparison without prompt engineering.

Result: Evaluated 30 advanced MLLMs, revealing significant room for improvement in current models and identifying potential optimization directions.

Conclusion: MME provides the first comprehensive evaluation framework for MLLMs, enabling fair comparisons and quantitative analysis while highlighting areas for future model development.

Abstract: Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image. However, it is difficult for these case studies to fully reflect the performance of MLLM, lacking a comprehensive evaluation. In this paper, we fill in this blank, presenting the first comprehensive MLLM Evaluation benchmark MME. It measures both perception and cognition abilities on a total of 14 subtasks. In order to avoid data leakage that may arise from direct use of public datasets for evaluation, the annotations of instruction-answer pairs are all manually designed. The concise instruction design allows us to fairly compare MLLMs, instead of struggling in prompt engineering. Besides, with such an instruction, we can also easily carry out quantitative statistics. A total of 30 advanced MLLMs are comprehensively evaluated on our MME, which not only suggests that existing MLLMs still have a large room for improvement, but also reveals the potential directions for the subsequent model optimization. The data are released at the project page https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Evaluation.

Method: Proposes Wavelet-context Cooperative Network (WCCNet) with cooperative dual-stream backbone: ADWT layers for lightweight infrared frequency extraction and neural layers for RGB features, plus crossmodal rearranging fusion module (CMRF) to handle spatial misalignment and merge complementary features.

Result: Experimental results on KAIST and FLIR benchmarks show WCCNet outperforms state-of-the-art methods with considerable efficiency and competitive accuracy.

Conclusion: WCCNet successfully addresses modality differences in multispectral pedestrian detection through differential feature extraction and effective crossmodal fusion, achieving superior performance with reduced computational complexity.

Abstract: Multispectral pedestrian detection achieves better visibility in challenging conditions and thus is essential to autonomous driving, for which both the accuracy and computational cost are of paramount importance. Most existing approaches treat RGB and infrared modalities equally. They typically adopt two symmetrical backbones for multimodal feature extraction, which ignore the substantial differences between modalities and bring great difficulty for the reduction of the computational cost as well as effective crossmodal fusion. In this work, we propose a novel and efficient framework named Wavelet-context Cooperative Network (WCCNet) that is able to differentially extract complementary features of different spectra with lower computational complexity, and further fuse these diverse features based on their spatially relevant crossmodal semantics. In particular, WCCNet simultaneously explore wavelet context and RGB textures within a cooperative dual-stream backbone, which is composed of adaptive discrete wavelet transform (ADWT) layers and heavyweight neural layers. The ADWT layers extract frequency components for infrared modality, while neural layers handle RGB modality features. Since ADWT layers are lightweight and extract complementary features, this cooperative structure not only significantly reduces the computational complexity, but also facilitates the subsequent crossmodal fusion. To further fuse these infrared and RGB features with significant semantic differences, we elaborately design the crossmodal rearranging fusion module (CMRF), which can mitigate spatial misalignment and merge semantically complementary features in spatially-related local regions to amplify the crossmodal reciprocal information. Experimental results on KAIST and FLIR benchmarks indicate that WCCNet outperforms state-of-the-art methods with considerable efficiency and competitive accuracy.

Method: Uses circle contour deformation with bounding circle-to-circle contour adaptation instead of bounding box-to-octagon transformation. Reduces degrees of freedom from 8 to 2, incorporates circular graph convolution, and provides a unified framework for circle detection, contour proposal, and circular convolution.

Result: Demonstrated superior performance and greater rotation invariance in practical applications including glomeruli, nuclei, and eosinophils detection in pathological images compared to benchmarks.

Conclusion: CircleSnake successfully integrates circle representation into an end-to-end deep instance segmentation pipeline, providing a robust and rotation-invariant solution for segmenting ball-shaped medical objects.

Abstract: Recently, circle representation has been introduced for medical imaging, designed specifically to enhance the detection of instance objects that are spherically shaped (e.g., cells, glomeruli, and nuclei). Given its outstanding effectiveness in instance detection, it is compelling to consider the application of circle representation for segmenting instance medical objects. In this study, we introduce CircleSnake, a simple end-to-end segmentation approach that utilizes circle contour deformation for segmenting ball-shaped medical objects at the instance level. The innovation of CircleSnake lies in these three areas: (1) It substitutes the complex bounding box-to-octagon contour transformation with a more consistent and rotation-invariant bounding circle-to-circle contour adaptation. This adaptation specifically targets ball-shaped medical objects. (2) The circle representation employed in CircleSnake significantly reduces the degrees of freedom to two, compared to eight in the octagon representation. This reduction enhances both the robustness of the segmentation performance and the rotational consistency of the method. (3) CircleSnake is the first end-to-end deep instance segmentation pipeline to incorporate circle representation, encompassing consistent circle detection, circle contour proposal, and circular convolution in a unified framework. This integration is achieved through the novel application of circular graph convolution within the context of circle detection and instance segmentation. In practical applications, such as the detection of glomeruli, nuclei, and eosinophils in pathological images, CircleSnake has demonstrated superior performance and greater rotation invariance when compared to benchmarks. The code has been made publicly available: https://github.com/hrlblab/CircleSnake.

Method: Proposes TopoFR with two strategies: PTSA uses persistent homology for topological structure alignment, and SDE identifies hard samples using structure damage score (SDS) to prioritize their optimization.

Result: Experimental results on popular face benchmarks demonstrate TopoFR’s superiority over state-of-the-art methods.

Conclusion: TopoFR effectively preserves structure information in latent space and improves face recognition performance through topological alignment and hard sample mining.

Abstract: The field of face recognition (FR) has undergone significant advancements with the rise of deep learning. Recently, the success of unsupervised learning and graph neural networks has demonstrated the effectiveness of data structure information. Considering that the FR task can leverage large-scale training data, which intrinsically contains significant structure information, we aim to investigate how to encode such critical structure information into the latent space. As revealed from our observations, directly aligning the structure information between the input and latent spaces inevitably suffers from an overfitting problem, leading to a structure collapse phenomenon in the latent space. To address this problem, we propose TopoFR, a novel FR model that leverages a topological structure alignment strategy called PTSA and a hard sample mining strategy named SDE. Concretely, PTSA uses persistent homology to align the topological structures of the input and latent spaces, effectively preserving the structure information and improving the generalization performance of FR model. To mitigate the impact of hard samples on the latent space structure, SDE accurately identifies hard samples by automatically computing structure damage score (SDS) for each sample, and directs the model to prioritize optimizing these samples. Experimental results on popular face benchmarks demonstrate the superiority of our TopoFR over the state-of-the-art methods. Code and models are available at: https://github.com/modelscope/facechain/tree/main/face_module/TopoFR.

Method: Decompose datasets into cue-specific versions, train cue experts on these reduced datasets, perform early fusion by constructing appropriate datasets, and implement late fusion of experts to study pixel-level cue influence.

Result: No single cue dominates; shape + color expert predominantly improves prediction of small objects and border pixels. Cue performance order is consistent across convolutional and transformer architectures, indicating similar cue extraction capabilities despite reported shape bias differences.

Conclusion: While pre-trained transformers are said to be more shape-biased than CNNs, both architectures show similar cue extraction capabilities during training, with shape+color combination being particularly beneficial for small objects and border regions.

Abstract: Recent research has investigated the shape and texture biases of pre-trained deep neural networks (DNNs) in image classification. Those works test how much a trained DNN relies on specific image cues like texture. The present study shifts the focus to understanding the cue influence during training, analyzing what DNNs can learn from shape, texture, and color cues in absence of the others; investigating their individual and combined influence on the learning success. We analyze these cue influences at multiple levels by decomposing datasets into cue-specific versions. Addressing semantic segmentation, we learn the given task from these reduced cue datasets, creating cue experts. Early fusion of cues is performed by constructing appropriate datasets. This is complemented by a late fusion of experts which allows us to study cue influence location-dependent on pixel level. Experiments on Cityscapes, PASCAL Context, and a synthetic CARLA dataset show that while no single cue dominates, the shape + color expert predominantly improves the prediction of small objects and border pixels. The cue performance order is consistent for the tested convolutional and transformer architecture, indicating similar cue extraction capabilities, although pre-trained transformers are said to be more biased towards shape than convolutional neural networks.

Method: Develops a novel encoding method that captures geometrical-topological characteristics of point clouds using inner products, creating a highly-efficient representation that can be integrated into deep learning models.

Result: The method exhibits high quality in typical tasks like reconstruction, generation, and interpolation, with inference times orders of magnitude faster than existing methods.

Conclusion: The inner product-based encoding provides an efficient and expressive representation for point clouds that significantly outperforms existing methods in speed while maintaining high quality.

Abstract: Point cloud synthesis, i.e. the generation of novel point clouds from an input distribution, remains a challenging task, for which numerous complex machine learning models have been devised. We develop a novel method that encodes geometrical-topological characteristics of point clouds using inner products, leading to a highly-efficient point cloud representation with provable expressivity properties. Integrated into deep learning models, our encoding exhibits high quality in typical tasks like reconstruction, generation, and interpolation, with inference times orders of magnitude faster than existing methods.

Method: Generate sub-motions from text descriptions, assemble them with random transitions into a coarse long sequence, then refine using SSD - an optimization-based method that aligns overlapping segments with a pre-trained motion diffusion prior.

Result: Extensive experiments show the framework can generate coherent, contextually aware motion sequences of arbitrary length, outperforming existing methods.

Conclusion: InfiniDreamer successfully addresses the long motion generation problem by leveraging short-clip motion priors in a training-free manner, enabling arbitrary-length motion synthesis.

Abstract: We present InfiniDreamer, a novel framework for arbitrarily long human motion generation. InfiniDreamer addresses the limitations of current motion generation methods, which are typically restricted to short sequences due to the lack of long motion training data. To achieve this, we first generate sub-motions corresponding to each textual description and then assemble them into a coarse, extended sequence using randomly initialized transition segments. We then introduce an optimization-based method called Segment Score Distillation (SSD) to refine the entire long motion sequence. SSD is designed to utilize an existing motion prior, which is trained only on short clips, in a training-free manner. Specifically, SSD iteratively refines overlapping short segments sampled from the coarsely extended long motion sequence, progressively aligning them with the pre-trained motion diffusion prior. This process ensures local coherence within each segment, while the refined transitions between segments maintain global consistency across the entire sequence. Extensive qualitative and quantitative experiments validate the superiority of our framework, showcasing its ability to generate coherent, contextually aware motion sequences of arbitrary length.

Method: Systematic analysis of MoE configurations and model scales across different architectures, testing parameter activation levels, expert placement strategies, and routing mechanisms.

Result: Moderate parameter activation provides best performance-efficiency trade-off; MoE most effective for tiny/mid-sized models; Last-2 placement is robust; larger datasets support more experts (up to 16); linear router performs best.

Conclusion: MoE layers effectively strengthen smaller vision models with practical design insights: optimal placement strategies, dataset-dependent expert scaling, and simple routing mechanisms provide efficient scaling without redefining SOTA performance.

Abstract: Mixture-of-Experts (MoE) models have shown promising potential for parameter-efficient scaling across domains. However, their application to image classification remains limited, often requiring billion-scale datasets to be competitive. In this work, we explore the integration of MoE layers into image classification architectures using open datasets. We conduct a systematic analysis across different MoE configurations and model scales. We find that moderate parameter activation per sample provides the best trade-off between performance and efficiency. However, as the number of activated parameters increases, the benefits of MoE diminish. Our analysis yields several practical insights for vision MoE design. First, MoE layers most effectively strengthen tiny and mid-sized models, while gains taper off for large-capacity networks and do not redefine state-of-the-art ImageNet performance. Second, a Last-2 placement heuristic offers the most robust cross-architecture choice, with Every-2 slightly better for Vision Transform (ViT), and both remaining effective as data and model scale increase. Third, larger datasets (e.g., ImageNet-21k) allow more experts, up to 16, for ConvNeXt to be utilized effectively without changing placement, as increased data reduces overfitting and promotes broader expert specialization. Finally, a simple linear router performs best, suggesting that additional routing complexity yields no consistent benefit.

Method: Compare native multimodal VLMs (trained from scratch for joint image-text generation) with non-native multimodal VLMs (adapted from pre-trained LLMs or text-only generators), analyzing information flow patterns and conducting ablation studies.

Result: Native VLMs show more separated image-text embeddings and rely on a single post-image token as a visual information gate. Ablating this token significantly reduces image-understanding performance, while targeted interventions can reliably steer image semantics and downstream text.

Conclusion: Different VLM architectures exhibit distinct visual information processing patterns, with native multimodal models using a narrow gating mechanism that enables fine-grained control over image semantics and text generation.

Abstract: Recent advances in multimodal training have significantly improved the integration of image understanding and generation within a unified model. This study investigates how vision-language models (VLMs) handle image-understanding tasks, focusing on how visual information is processed and transferred to the textual domain. We compare native multimodal VLMs, models trained from scratch on multimodal data to generate both text and images, and non-native multimodal VLMs, models adapted from pre-trained large language models or capable of generating only text, highlighting key differences in information flow. We find that in native multimodal VLMs, image and text embeddings are more separated within the residual stream. Moreover, VLMs differ in how visual information reaches text: non-native multimodal VLMs exhibit a distributed communication pattern, where information is exchanged through multiple image tokens, whereas models trained natively for joint image and text generation tend to rely on a single post-image token that acts as a narrow gate for visual information. We show that ablating this single token significantly deteriorates image-understanding performance, whereas targeted, token-level interventions reliably steer image semantics and downstream text with fine-grained control.

Method: Uses residual-guided adversarial pattern exploration to address noisy feedback, and distribution-matched attack scenario alignment with conditional-uncertainty-aligned data and skewness-aligned objective re-weighting modules to align training with real-world conditions.

Result: Achieves 2.07x boost over state-of-the-art static PAE methods with 58.8% average AP drop on object detectors like DETR, demonstrating superior attack performance in both digital and physical evaluations.

Conclusion: DynamicPAE enables end-to-end modeling of dynamic physical adversarial examples, opening new possibilities for real-time scene-aware attacks in varying environments.

Abstract: Physical adversarial examples (PAEs) are regarded as whistle-blowers of real-world risks in deep-learning applications, thus worth further investigation. However, current PAE generation studies show limited adaptive attacking ability to diverse and varying scenes, revealing the urgent requirement of dynamic PAEs that are generated in real time and conditioned on the observation from the attacker. The key challenge in generating dynamic PAEs is learning the sparse relation between PAEs and the observation of attackers under the noisy feedback of attack training. To address the challenge, we present DynamicPAE, the first generative framework that enables scene-aware real-time physical attacks. Specifically, to address the noisy feedback problem that obfuscates the exploration of scene-related PAEs, we introduce the residual-guided adversarial pattern exploration technique. Residual-guided training, which relaxes the attack training with a reconstruction task, is proposed to enrich the feedback information, thereby achieving a more comprehensive exploration of PAEs. To address the alignment problem between the trained generator and the real-world scenario, we introduce the distribution-matched attack scenario alignment, consisting of the conditional-uncertainty-aligned data module and the skewness-aligned objective re-weighting module. The former aligns the training environment with the incomplete observation of the real-world attacker. The latter facilitates consistent stealth control across different attack targets with the skewness controller. Extensive digital and physical evaluations demonstrate the superior attack performance of DynamicPAE, attaining a 2.07 $\times$ boost (58.8% average AP drop under attack) on representative object detectors (e.g., DETR) over state-of-the-art static PAE generating methods. Overall, our work opens the door to end-to-end modeling of dynamic PAEs.

Method: Proposes Distilled Decoding (DD) which uses flow matching to create a deterministic mapping from Gaussian distribution to the AR model’s output distribution, then trains a network to distill this mapping for few-step generation without needing original training data.

Result: DD enables one-step generation for VAR (6.3× speed-up, FID 4.19→9.96) and reduces LlamaGen from 256 to 1 step (217.8× speed-up, FID 4.11→11.35). Baseline methods fail completely (FID>100). Also works well for text-to-image generation.

Conclusion: DD challenges the notion that AR models are inherently slow and opens new opportunities for efficient AR generation, demonstrating the first successful one-step generation for image AR models.

Abstract: Autoregressive (AR) models have achieved state-of-the-art performance in text and image generation but suffer from slow generation due to the token-by-token process. We ask an ambitious question: can a pre-trained AR model be adapted to generate outputs in just one or two steps? If successful, this would significantly advance the development and deployment of AR models. We notice that existing works that try to speed up AR generation by generating multiple tokens at once fundamentally cannot capture the output distribution due to the conditional dependencies between tokens, limiting their effectiveness for few-step generation. To address this, we propose Distilled Decoding (DD), which uses flow matching to create a deterministic mapping from Gaussian distribution to the output distribution of the pre-trained AR model. We then train a network to distill this mapping, enabling few-step generation. DD doesn’t need the training data of the original AR model, making it more practical. We evaluate DD on state-of-the-art image AR models and present promising results on ImageNet-256. For VAR, which requires 10-step generation, DD enables one-step generation (6.3$\times$ speed-up), with an acceptable increase in FID from 4.19 to 9.96. For LlamaGen, DD reduces generation from 256 steps to 1, achieving an 217.8$\times$ speed-up with a comparable FID increase from 4.11 to 11.35. In both cases, baseline methods completely fail with FID>100. DD also excels on text-to-image generation, reducing the generation from 256 steps to 2 for LlamaGen with minimal FID increase from 25.70 to 28.95. As the first work to demonstrate the possibility of one-step generation for image AR models, DD challenges the prevailing notion that AR models are inherently slow, and opens up new opportunities for efficient AR generation. The project website is at https://imagination-research.github.io/distilled-decoding.

Method: Spatial Adversarial Alignment (SAA) uses alignment loss with a witness model to fine-tune surrogate models through spatial-aware alignment (global/local feature divergence minimization) and adversarial-aware alignment (self-adversarial strategy).

Result: Extensive experiments on ImageNet show SAA-aligned surrogate models generate more transferable adversarial examples, particularly in cross-architecture attacks.

Conclusion: SAA effectively improves adversarial transferability by training surrogate models to focus on common features shared with witness models.

Abstract: Deep neural networks are vulnerable to adversarial examples that exhibit transferability across various models. Numerous approaches are proposed to enhance the transferability of adversarial examples, including advanced optimization, data augmentation, and model modifications. However, these methods still show limited transferability, particularly in cross-architecture scenarios, such as from CNN to ViT. To achieve high transferability, we propose a technique termed Spatial Adversarial Alignment (SAA), which employs an alignment loss and leverages a witness model to fine-tune the surrogate model. Specifically, SAA consists of two key parts: spatial-aware alignment and adversarial-aware alignment. First, we minimize the divergences of features between the two models in both global and local regions, facilitating spatial alignment. Second, we introduce a self-adversarial strategy that leverages adversarial examples to impose further constraints, aligning features from an adversarial perspective. Through this alignment, the surrogate model is trained to concentrate on the common features extracted by the witness model. This facilitates adversarial attacks on these shared features, thereby yielding perturbations that exhibit enhanced transferability. Extensive experiments on various architectures on ImageNet show that aligned surrogate models based on SAA can provide higher transferable adversarial examples, especially in cross-architecture attacks.

Method: Uses multi-scale dynamic visual-skeleton encoder, procedure segmentation network, multi-modal fusion with physics structural priors, and multi-stage contrastive learning regression.

Result: Demonstrates effectiveness and superiority on FineDiving and MTL-AQA datasets, with improved performance over existing methods.

Conclusion: The proposed approach successfully addresses key challenges in AQA by capturing fine-grained pose differences and maintaining temporal continuity, with a new FineDiving-Pose Dataset provided for better pose labels.

Abstract: Action Quality Assessment (AQA), which aims at automatic and fair evaluation of athletic performance, has gained increasing attention in recent years. However, athletes are often in rapid movement and the corresponding visual appearance variances are subtle, making it challenging to capture fine-grained pose differences and leading to poor estimation performance. Furthermore, most common AQA tasks, such as diving in sports, are usually divided into multiple sub-actions, each of which contains different durations. However, existing methods focus on segmenting the video into fixed frames, which disrupts the temporal continuity of sub-actions resulting in unavoidable prediction errors. To address these challenges, we propose a novel action quality assessment method through hierarchically pose-guided multi-stage contrastive regression. Firstly, we introduce a multi-scale dynamic visual-skeleton encoder to capture fine-grained spatio-temporal visual and skeletal features. Then, a procedure segmentation network is introduced to separate different sub-actions and obtain segmented features. Afterwards, the segmented visual and skeletal features are both fed into a multi-modal fusion module as physics structural priors, to guide the model in learning refined activity similarities and variances. Finally, a multi-stage contrastive learning regression approach is employed to learn discriminative representations and output prediction results. In addition, we introduce a newly-annotated FineDiving-Pose Dataset to improve the current low-quality human pose labels. In experiments, the results on FineDiving and MTL-AQA datasets demonstrate the effectiveness and superiority of our proposed approach. Our source code and dataset are available at https://github.com/Lumos0507/HP-MCoRe.

Method: Uses autoregressive transformer with BFS-ordered joints, treating skeleton as sequence of 3D locations and parent indices, enhanced with diffusion modeling for precise position prediction.

Result: Achieves state-of-the-art performance across humanoids, quadrupeds, marine creatures, insects, etc., with significantly faster rigging (under few seconds per shape) and better quality, robustness, and generalizability.

Conclusion: RigAnything demonstrates effective template-free rigging through autoregressive modeling of tree structures, enabling broad applicability and superior performance compared to prior methods.

Abstract: We present RigAnything, a novel autoregressive transformer-based model, which makes 3D assets rig-ready by probabilistically generating joints and skeleton topologies and assigning skinning weights in a template-free manner. Unlike most existing auto-rigging methods, which rely on predefined skeleton templates and are limited to specific categories like humanoid, RigAnything approaches the rigging problem in an autoregressive manner, iteratively predicting the next joint based on the global input shape and the previous prediction. While autoregressive models are typically used to generate sequential data, RigAnything extends its application to effectively learn and represent skeletons, which are inherently tree structures. To achieve this, we organize the joints in a breadth-first search (BFS) order, enabling the skeleton to be defined as a sequence of 3D locations and the parent index. Furthermore, our model improves the accuracy of position prediction by leveraging diffusion modeling, ensuring precise and consistent placement of joints within the hierarchy. This formulation allows the autoregressive model to efficiently capture both spatial and hierarchical relationships within the skeleton. Trained end-to-end on both RigNet and Objaverse datasets, RigAnything demonstrates state-of-the-art performance across diverse object types, including humanoids, quadrupeds, marine creatures, insects, and many more, surpassing prior methods in quality, robustness, generalizability, and efficiency. It achieves significantly faster performance than existing auto-rigging methods, completing rigging in under a few seconds per shape. Please check our website for more details: https://www.liuisabella.com/RigAnything

Method: UniTok features a novel multi-codebook quantization mechanism that effectively scales up vocabulary size and bottleneck dimension, addressing the limited representational capacity issue. It can be seamlessly integrated into multimodal large language models (MLLMs).

Result: UniTok sets new records with 0.38 rFID and 78.6% zero-shot accuracy on ImageNet. It enables cfg-free generation, reducing gFID from 14.6 to 2.5 on ImageNet 256×256 benchmark. It unlocks native visual generation capability in MLLMs without compromising understanding performance.

Conclusion: Reconstruction and semantic supervision do not inherently conflict; the bottleneck is limited representational capacity of discrete token space. UniTok’s multi-codebook quantization effectively addresses this limitation, enabling unified tokenization for both generation and understanding tasks.

Abstract: Visual generative and understanding models typically rely on distinct tokenizers to process images, presenting a key challenge for unifying them within a single framework. Recent studies attempt to address this by connecting the training of VQVAE (for autoregressive generation) and CLIP (for understanding) to build a unified tokenizer. However, directly combining these training objectives has been observed to cause severe loss conflicts. In this paper, we show that reconstruction and semantic supervision do not inherently conflict. Instead, the underlying bottleneck stems from limited representational capacity of discrete token space. Building on these insights, we introduce UniTok, a unified tokenizer featuring a novel multi-codebook quantization mechanism that effectively scales up the vocabulary size and bottleneck dimension. In terms of final performance, UniTok sets a new record of 0.38 rFID and 78.6% zero-shot accuracy on ImageNet. Besides, UniTok can be seamlessly integrated into MLLMs to unlock native visual generation capability, without compromising the understanding performance. Additionally, we show that UniTok favors cfg-free generation, reducing gFID from 14.6 to 2.5 on ImageNet 256$\times$256 benchmark. GitHub: https://github.com/FoundationVision/UniTok.

Method: Strategic sampling from class-conditional generative models trained exclusively on target face recognition datasets, eliminating need for external resources.

Result: Achieves 1-12% performance improvements across 8 FR benchmarks (IJB-C, IJB-B), outperforms real-data-only training and state-of-the-art synthetic approaches while using less real data.

Conclusion: Carefully integrated synthetic data can mitigate privacy constraints and substantially enhance recognition performance, with gains often exceeding architectural enhancements in data-limited scenarios.

Abstract: The increasing reliance on large-scale datasets in machine learning poses significant privacy and ethical challenges, particularly in sensitive domains such as face recognition. Synthetic data generation offers a promising alternative; however, most existing methods depend heavily on external datasets or pre-trained models, increasing complexity and resource demands. In this paper, we introduce AugGen, a self-contained synthetic augmentation technique. AugGen strategically samples from a class-conditional generative model trained exclusively on the target FR dataset, eliminating the need for external resources. Evaluated across 8 FR benchmarks, including IJB-C and IJB-B, our method achieves 1-12% performance improvements, outperforming models trained solely on real data and surpassing state-of-the-art synthetic data generation approaches, while using less real data. Notably, these gains often exceed those from architectural enhancements, underscoring the value of synthetic augmentation in data-limited scenarios. Our findings demonstrate that carefully integrated synthetic data can both mitigate privacy constraints and substantially enhance recognition performance. Paper website: https://parsa-ra.github.io/auggen/.

Method: Proposes LEGNet with Edge-Gaussian Aggregation (EGA) module that integrates orientation-aware Scharr filters to sharpen edge details and Gaussian-prior-based feature refinement to suppress noise and regularize ambiguous feature responses.

Result: Comprehensive evaluations across five benchmarks (DOTA-v1.0, v1.5, DIOR-R, FAIR1M-v1.0, and VisDrone2019) demonstrate state-of-the-art performance, particularly in detecting low-quality objects.

Conclusion: LEGNet effectively improves model robustness while maintaining computational efficiency, addressing prevalent problems in degraded remote sensing images such as reduced contrast, structural discontinuities, and ambiguous feature responses.

Abstract: Remote sensing object detection (RSOD) often suffers from degradations such as low spatial resolution, sensor noise, motion blur, and adverse illumination. These factors diminish feature distinctiveness, leading to ambiguous object representations and inadequate foreground-background separation. Existing RSOD methods exhibit limitations in robust detection of low-quality objects. To address these pressing challenges, we introduce LEGNet, a lightweight backbone network featuring a novel Edge-Gaussian Aggregation (EGA) module specifically engineered to enhance feature representation derived from low-quality remote sensing images. EGA module integrates: (a) orientation-aware Scharr filters to sharpen crucial edge details often lost in low-contrast or blurred objects, and (b) Gaussian-prior-based feature refinement to suppress noise and regularize ambiguous feature responses, enhancing foreground saliency under challenging conditions. EGA module alleviates prevalent problems in reduced contrast, structural discontinuities, and ambiguous feature responses prevalent in degraded images, effectively improving model robustness while maintaining computational efficiency. Comprehensive evaluations across five benchmarks (DOTA-v1.0, v1.5, DIOR-R, FAIR1M-v1.0, and VisDrone2019) demonstrate that LEGNet achieves state-of-the-art performance, particularly in detecting low-quality objects.The code is available at https://github.com/AeroVILab-AHU/LEGNet.

Method: Comprehensive review and categorization of automatic change detection methods into four families: rule-based, statistical, machine learning, and simulation methods, analyzing their strengths, limitations, and applicability across different input data types.

Result: Identified key applications for National Mapping Agencies including geospatial database optimization, change-based phenomena monitoring, and dynamics tracking, while highlighting current challenges in change detection implementation.

Conclusion: Ongoing innovation in change detection techniques is essential to address future needs of geographic information systems for national mapping agencies, particularly given operational constraints and evolving data requirements.

Abstract: Rapid evolution of territories due to climate change and human impact requires prompt and effective updates to geospatial databases maintained by the National Mapping Agency. This paper presents a comprehensive overview of change detection methods tailored for the operational updating of large-scale geographic databases. This review first outlines the fundamental definition of change, emphasizing its multifaceted nature, from temporal to semantic characterization. It categorizes automatic change detection methods into four main families: rule-based, statistical, machine learning, and simulation methods. The strengths, limitations, and applicability of every family are discussed in the context of various input data. Then, key applications for National Mapping Agencies are identified, particularly the optimization of geospatial database updating, change-based phenomena, and dynamics monitoring. Finally, the paper highlights the current challenges for leveraging change detection such as the variability of change definition, the missing of relevant large-scale datasets, the diversity of input data, the unstudied no-change detection, the human in the loop integration and the operational constraints. The discussion underscores the necessity for ongoing innovation in change detection techniques to address the future needs of geographic information systems for national mapping agencies.

Method: Three main components: 1) SDE-based generation to enable particle sampling, 2) Interpolant conversion to expand search space and enhance diversity, 3) Rollover Budget Forcing (RBF) for adaptive computational resource allocation across timesteps.

Result: SDE-based generation, especially variance-preserving (VP) interpolant-based generation, improves particle sampling performance. RBF with VP-SDE achieves the best performance, outperforming all previous inference-time scaling approaches.

Conclusion: The proposed method successfully enables efficient inference-time scaling for flow models, overcoming their deterministic limitations and achieving superior performance compared to existing approaches.

Abstract: We propose an inference-time scaling approach for pretrained flow models. Recently, inference-time scaling has gained significant attention in LLMs and diffusion models, improving sample quality or better aligning outputs with user preferences by leveraging additional computation. For diffusion models, particle sampling has allowed more efficient scaling due to the stochasticity at intermediate denoising steps. On the contrary, while flow models have gained popularity as an alternative to diffusion models–offering faster generation and high-quality outputs in state-of-the-art image and video generative models–efficient inference-time scaling methods used for diffusion models cannot be directly applied due to their deterministic generative process. To enable efficient inference-time scaling for flow models, we propose three key ideas: 1) SDE-based generation, enabling particle sampling in flow models, 2) Interpolant conversion, broadening the search space and enhancing sample diversity, and 3) Rollover Budget Forcing (RBF), an adaptive allocation of computational resources across timesteps to maximize budget utilization. Our experiments show that SDE-based generation, particularly variance-preserving (VP) interpolant-based generation, improves the performance of particle sampling methods for inference-time scaling in flow models. Additionally, we demonstrate that RBF with VP-SDE achieves the best performance, outperforming all previous inference-time scaling approaches.

Method: Developed a degraded imaging model based on Retinex theory and atmospheric scattering to simulate composite degradations, and created a prompt-modulated restoration and fusion network with degradation prompts. Includes text encoder for user-specified degradation types/levels and spatial-frequency collaborative visual adapter for autonomous degradation perception.

Result: Extensive experiments show ControlFusion outperforms state-of-the-art fusion methods in fusion quality and degradation handling, especially for real-world and compound degradations at various levels.

Conclusion: ControlFusion provides an effective solution for handling composite degradations in image fusion with user-controllable parameters, demonstrating superior performance particularly in challenging real-world scenarios.

Abstract: Current image fusion methods struggle to address the composite degradations encountered in real-world imaging scenarios and lack the flexibility to accommodate user-specific requirements. In response to these challenges, we propose a controllable image fusion framework with language-vision prompts, termed ControlFusion, which adaptively neutralizes composite degradations. On the one hand, we develop a degraded imaging model that integrates physical imaging mechanisms, including the Retinex theory and atmospheric scattering principle, to simulate composite degradations, thereby providing potential for addressing real-world complex degradations from the data level. On the other hand, we devise a prompt-modulated restoration and fusion network that dynamically enhances features with degradation prompts, enabling our method to accommodate composite degradation of varying levels. Specifically, considering individual variations in quality perception of users, we incorporate a text encoder to embed user-specified degradation types and severity levels as degradation prompts. We also design a spatial-frequency collaborative visual adapter that autonomously perceives degradations in source images, thus eliminating the complete dependence on user instructions. Extensive experiments demonstrate that ControlFusion outperforms SOTA fusion methods in fusion quality and degradation handling, particularly in countering real-world and compound degradations with various levels. The source code is publicly available at https://github.com/Linfeng-Tang/ControlFusion.

Method: Built on RT-DETR base detector with three key components: local object-level feature alignment module, scene semantic feature alignment module, and domain query decoupled from object query in decoder layers.

Result: Outperforms current state-of-the-art approaches on various cross-domain benchmarks.

Conclusion: RT-DATR provides an effective solution for real-time domain adaptive object detection using transformers, with demonstrated superior performance over existing methods.

Abstract: Despite domain-adaptive object detectors based on CNN and transformers have made significant progress in cross-domain detection tasks, it is regrettable that domain adaptation for real-time transformer-based detectors has not yet been explored. Directly applying existing domain adaptation algorithms has proven to be suboptimal. In this paper, we propose RT-DATR, a simple and efficient real-time domain adaptive detection transformer. Building on RT-DETR as our base detector, we first introduce a local object-level feature alignment module to significantly enhance the feature representation of domain invariance during object transfer. Additionally, we introduce a scene semantic feature alignment module designed to boost cross-domain detection performance by aligning scene semantic features. Finally, we introduced a domain query and decoupled it from the object query to further align the instance feature distribution within the decoder layer, reduce the domain gap, and maintain discriminative ability. Experimental results on various cross-domian benchmarks demonstrate that our method outperforms current state-of-the-art approaches. Code is available at https://github.com/Jeremy-lf/RT-DATR.

Method: Proposed HAVT-IVD network with heterogeneity-aware architecture, visual feature pyramid for multi-resolution detection, and decoupled heads to address training instability.

Result: Experiments show HAVT-IVD improves mAP by 7.66 over disjoint baseline and 9.42 over E2E baseline.

Conclusion: HAVT-IVD effectively addresses the key challenges in idling vehicle detection through its heterogeneity-aware design, achieving substantial performance improvements over existing approaches.

Abstract: Idling vehicle detection (IVD) uses surveillance video and multichannel audio to localize and classify vehicles in the last frame as moving, idling, or engine-off in pick-up zones. IVD faces three challenges: (i) modality heterogeneity between visual cues and audio patterns; (ii) large box scale variation requiring multi-resolution detection; and (iii) training instability due to coupled detection heads. The previous end-to-end (E2E) model with simple CBAM-based bi-modal attention fails to handle these issues and often misses vehicles. We propose HAVT-IVD, a heterogeneity-aware network with a visual feature pyramid and decoupled heads. Experiments show HAVT-IVD improves mAP by 7.66 over the disjoint baseline and 9.42 over the E2E baseline.

Method: SPORT has four iterative components: task synthesis using language models, step sampling (trying different tools), step verification (AI feedback), and preference tuning to update the controller. The agent autonomously explores tool usage strategies through self-exploration in real environments.

Result: Evaluation on GTA and GAIA benchmarks shows SPORT agent achieves 6.41% and 3.64% improvements respectively, demonstrating strong generalization and effectiveness.

Conclusion: SPORT enables multimodal agents to autonomously discover effective tool usage strategies through self-exploration and optimization, eliminating the bottleneck of human annotation while achieving significant performance improvements.

Abstract: Multimodal agents, which integrate a controller e.g., a vision language model) with external tools, have demonstrated remarkable capabilities in tackling complex multimodal tasks. Existing approaches for training these agents, both supervised fine-tuning and reinforcement learning, depend on extensive human-annotated task-answer pairs and tool trajectories. However, for complex multimodal tasks, such annotations are prohibitively expensive or impractical to obtain. In this paper, we propose an iterative tool usage exploration method for multimodal agents without any pre-collected data, namely SPORT, via step-wise preference optimization to refine the trajectories of tool usage. Our method enables multimodal agents to autonomously discover effective tool usage strategies through self-exploration and optimization, eliminating the bottleneck of human annotation. SPORT has four iterative components: task synthesis, step sampling, step verification, and preference tuning. We first synthesize multimodal tasks using language models. Then, we introduce a novel trajectory exploration scheme, where step sampling and step verification are executed alternately to solve synthesized tasks. In step sampling, the agent tries different tools and obtains corresponding results. In step verification, we employ a verifier to provide AI feedback to construct step-wise preference data. The data is subsequently used to update the controller for tool usage through preference tuning, producing a SPORT agent. By interacting with real environments, the SPORT agent gradually evolves into a more refined and capable system. Evaluation in the GTA and GAIA benchmarks shows that the SPORT agent achieves 6.41% and 3.64% improvements, underscoring the generalization and effectiveness introduced by our method. The project page is https://SPORT-Agents.github.io.

Method: RTV-Bench uses three key principles: Multi-Timestamp Question Answering (MTQA) with evolving answers, hierarchical question structure combining basic and advanced queries, and multi-dimensional evaluation of continuous perception, understanding, and reasoning.

Result: Open-source real-time models largely outperform offline ones but still trail top proprietary models. Larger model size or higher frame sampling rates do not significantly boost performance, sometimes causing slight decreases.

Conclusion: There is a need for better model architectures optimized for video stream processing and long sequences to advance real-time video analysis with MLLMs.

Abstract: Multimodal Large Language Models (MLLMs) increasingly excel at perception, understanding, and reasoning. However, current benchmarks inadequately evaluate their ability to perform these tasks continuously in dynamic, real-world environments. To bridge this gap, we introduce RTV-Bench, a fine-grained benchmark for MLLM real-time video analysis. RTV-Bench uses three key principles: (1) Multi-Timestamp Question Answering (MTQA), where answers evolve with scene changes; (2) Hierarchical Question Structure, combining basic and advanced queries; and (3) Multi-dimensional Evaluation, assessing the ability of continuous perception, understanding, and reasoning. RTV-Bench contains 552 diverse videos (167.2 hours) and 4,631 high-quality QA pairs. We evaluated leading MLLMs, including proprietary (GPT-4o, Gemini 2.0), open-source offline (Qwen2.5-VL, VideoLLaMA3), and open-source real-time (VITA-1.5, InternLM-XComposer2.5-OmniLive) models. Experiment results show open-source real-time models largely outperform offline ones but still trail top proprietary models. Our analysis also reveals that larger model size or higher frame sampling rates do not significantly boost RTV-Bench performance, sometimes causing slight decreases. This underscores the need for better model architectures optimized for video stream processing and long sequences to advance real-time video analysis with MLLMs. Our benchmark toolkit is available at: https://github.com/LJungang/RTV-Bench.

Method: Examine influence of register tokens on global-local feature relationships, analyze attention maps, and compare with [CLS] token behavior in models without registers.

Result: Register tokens yield cleaner attention maps but don’t accurately reflect local information integration; global information is dominated by register/CLS tokens, creating local-global feature disconnect.

Conclusion: Care needed when interpreting large ViT attention maps; identifying register and CLS tokens as sources of faulty behavior provides path to more interpretable vision models.

Abstract: Recent work has shown that the attention maps of the widely popular DINOv2 model exhibit artifacts, which hurt both model interpretability and performance on dense image tasks. These artifacts emerge due to the model repurposing patch tokens with redundant local information for the storage of global image information. To address this problem, additional register tokens have been incorporated in which the model can store such information instead. We carefully examine the influence of these register tokens on the relationship between global and local image features, showing that while register tokens yield cleaner attention maps, these maps do not accurately reflect the integration of local image information in large models. Instead, global information is dominated by information extracted from register tokens, leading to a disconnect between local and global features. Inspired by these findings, we show that the [CLS] token itself leads to a very similar phenomenon in models without explicit register tokens. Our work shows that care must be taken when interpreting attention maps of large ViTs. Further, by clearly attributing the faulty behavior to register and [CLS] tokens, we show a path towards more interpretable vision models.

Method: Proposed MELLM with MEFlowNet - an iterative warping-based optical flow estimator to capture facial micro-movements. Created MEFlowDataset (54,611 image pairs) and MEU-Instruct dataset. Uses flow-guided paradigm to translate motion patterns into descriptions and emotional inferences.

Result: MEFlowNet significantly outperforms existing optical flow methods in facial and ME-flow estimation. MELLM achieves state-of-the-art accuracy and generalization across multiple ME benchmarks.

Conclusion: This work presents two key contributions: MEFlowNet as the first dedicated ME flow estimator, and MELLM as the first LLM tailored for comprehensive micro-expression understanding.

Abstract: Micro-expressions (MEs), brief and low-intensity facial movements revealing concealed emotions, are crucial for affective computing. Despite notable progress in ME recognition, existing methods are largely confined to discrete emotion classification, lacking the capacity for comprehensive ME Understanding (MEU), particularly in interpreting subtle facial dynamics and underlying emotional cues. While Multimodal Large Language Models (MLLMs) offer potential for MEU with their advanced reasoning abilities, they still struggle to perceive such subtle facial affective behaviors. To bridge this gap, we propose a ME Large Language Model (MELLM) that integrates optical flow-based sensitivity to subtle facial motions with the powerful inference ability of LLMs. Specifically, an iterative, warping-based optical-flow estimator, named MEFlowNet, is introduced to precisely capture facial micro-movements. For its training and evaluation, we construct MEFlowDataset, a large-scale optical-flow dataset with 54,611 onset-apex image pairs spanning diverse identities and subtle facial motions. Subsequently, we design a Flow-Guided Micro-Expression Understanding paradigm. Under this framework, the optical flow signals extracted by MEFlowNet are leveraged to build MEU-Instruct, an instruction-tuning dataset for MEU. MELLM is then fine-tuned on MEU-Instruct, enabling it to translate subtle motion patterns into human-readable descriptions and generate corresponding emotional inferences. Experiments demonstrate that MEFlowNet significantly outperforms existing optical flow methods in facial and ME-flow estimation, while MELLM achieves state-of-the-art accuracy and generalization across multiple ME benchmarks. To the best of our knowledge, this work presents two key contributions: MEFlowNet as the first dedicated ME flow estimator, and MELLM as the first LLM tailored for MEU.

Method: Uses pretrained teacher embedding model to rank examples, constructs sparse similarity graph, applies community detection to identify clusters of strong negatives, then constructs batches from these clusters.

Result: Sets new SOTA on MMEB benchmark (36 tasks): +1.3 points at 7B scale, +2.9 points at 2B scale. Achieves strong performance with batch size 64 (4-16x smaller than other methods). Generalizes well across domains and maintains performance with weaker teachers.

Conclusion: B3 effectively breaks the batch size barrier in contrastive learning by intelligently constructing high-quality batches, enabling superior performance with significantly smaller computational requirements.

Abstract: Contrastive learning (CL) is a prevalent technique for training embedding models, which pulls semantically similar examples (positives) closer in the representation space while pushing dissimilar ones (negatives) further apart. A key source of negatives are ‘in-batch’ examples, i.e., positives from other examples in the batch. Effectiveness of such models is hence strongly influenced by the size and quality of training batches. In this work, we propose ‘Breaking the Batch Barrier’ (B3), a novel batch construction strategy designed to curate high-quality batches for CL. Our approach begins by using a pretrained teacher embedding model to rank all examples in the dataset, from which a sparse similarity graph is constructed. A community detection algorithm is then applied to this graph to identify clusters of examples that serve as strong negatives for one another. The clusters are then used to construct batches that are rich in in-batch negatives. Empirical results on the MMEB multimodal embedding benchmark (36 tasks) demonstrate that our method sets a new state of the art, outperforming previous best methods by +1.3 and +2.9 points at the 7B and 2B model scales, respectively. Notably, models trained with B3 surpass existing state-of-the-art results even with a batch size as small as 64, which is 4-16x smaller than that required by other methods. Moreover, experiments show that B3 generalizes well across domains and tasks, maintaining strong performance even when trained with considerably weaker teachers.

Method: Transform raw depth data into structured textual rationales as intermediate representations, then use knowledge distillation to compress them into compact latent embeddings for plug-and-play integration into VLMs.

Result: Extensive experiments show SSR substantially improves depth utilization and enhances spatial reasoning, advancing VLMs toward more human-like multi-modal understanding.

Conclusion: SSR effectively enhances spatial reasoning in VLMs through structured depth-to-text transformations and efficient integration via knowledge distillation.

Abstract: Despite impressive advancements in Visual-Language Models (VLMs) for multi-modal tasks, their reliance on RGB inputs limits precise spatial understanding. Existing methods for integrating spatial cues, such as point clouds or depth, either require specialized sensors or fail to effectively exploit depth information for higher-order reasoning. To this end, we propose a novel Spatial Sense and Reasoning method, dubbed SSR, a novel framework that transforms raw depth data into structured, interpretable textual rationales. These textual rationales serve as meaningful intermediate representations to significantly enhance spatial reasoning capabilities. Additionally, we leverage knowledge distillation to compress the generated rationales into compact latent embeddings, which facilitate resource-efficient and plug-and-play integration into existing VLMs without retraining. To enable comprehensive evaluation, we introduce a new dataset named SSR-CoT, a million-scale visual-language reasoning dataset enriched with intermediate spatial reasoning annotations, and present SSRBench, a comprehensive multi-task benchmark. Extensive experiments on multiple benchmarks demonstrate SSR substantially improves depth utilization and enhances spatial reasoning, thereby advancing VLMs toward more human-like multi-modal understanding. Project page: https://yliu-cs.github.io/SSR.

Method: Two-phase training: 1) Instruction tuning on synthesized reasoning traces to familiarize models with visual operations, 2) Reinforcement learning with curiosity-driven reward scheme to balance pixel-space and textual reasoning.

Result: The 7B model achieves 84% on V* bench, 74% on TallyQA-Complex, and 84% on InfographicsVQA, marking the highest accuracy by any open-source model to date.

Conclusion: Pixel-space reasoning significantly improves VLM performance across diverse visual reasoning benchmarks, demonstrating the importance and effectiveness of the proposed framework.

Abstract: Chain-of-thought reasoning has significantly improved the performance of Large Language Models (LLMs) across various domains. However, this reasoning process has been confined exclusively to textual space, limiting its effectiveness in visually intensive tasks. To address this limitation, we introduce the concept of reasoning in the pixel-space. Within this novel framework, Vision-Language Models (VLMs) are equipped with a suite of visual reasoning operations, such as zoom-in and select-frame. These operations enable VLMs to directly inspect, interrogate, and infer from visual evidences, thereby enhancing reasoning fidelity for visual tasks. Cultivating such pixel-space reasoning capabilities in VLMs presents notable challenges, including the model’s initially imbalanced competence and its reluctance to adopt the newly introduced pixel-space operations. We address these challenges through a two-phase training approach. The first phase employs instruction tuning on synthesized reasoning traces to familiarize the model with the novel visual operations. Following this, a reinforcement learning (RL) phase leverages a curiosity-driven reward scheme to balance exploration between pixel-space reasoning and textual reasoning. With these visual operations, VLMs can interact with complex visual inputs, such as information-rich images or videos to proactively gather necessary information. We demonstrate that this approach significantly improves VLM performance across diverse visual reasoning benchmarks. Our 7B model, \model, achieves 84% on V* bench, 74% on TallyQA-Complex, and 84% on InfographicsVQA, marking the highest accuracy achieved by any open-source model to date. These results highlight the importance of pixel-space reasoning and the effectiveness of our framework.

Method: Constructs unified graph with parent-child links between coarse (5x) and fine (20x) visual/textual nodes, heterogeneous intra-scale edges linking visual and textual nodes, two-stage text-guided dynamic filtering to remove weakly correlated patch-text pairs, and hierarchical contrastive loss for semantic alignment.

Result: Outperforms traditional MIL and recent VLM-based MIL approaches on TCGA breast, lung, and kidney cancer datasets, achieving gains of up to 4.1% in macro F1 under 16-shot settings.

Conclusion: Jointly modeling hierarchical structure and multimodal alignment enables efficient and scalable learning from limited pathology data, demonstrating the value of the proposed approach.

Abstract: Vision-language models (VLMs) have recently been integrated into multiple instance learning (MIL) frameworks to address the challenge of few-shot, weakly supervised classification of whole slide images (WSIs). A key trend involves leveraging multi-scale information to better represent hierarchical tissue structures. However, existing methods often face two key limitations: (1) insufficient modeling of interactions within the same modalities across scales (e.g., 5x and 20x) and (2) inadequate alignment between visual and textual modalities on the same scale. To address these gaps, we propose HiVE-MIL, a hierarchical vision-language framework that constructs a unified graph consisting of (1) parent-child links between coarse (5x) and fine (20x) visual/textual nodes to capture hierarchical relationships, and (2) heterogeneous intra-scale edges linking visual and textual nodes on the same scale. To further enhance semantic consistency, HiVE-MIL incorporates a two-stage, text-guided dynamic filtering mechanism that removes weakly correlated patch-text pairs, and introduces a hierarchical contrastive loss to align textual semantics across scales. Extensive experiments on TCGA breast, lung, and kidney cancer datasets demonstrate that HiVE-MIL consistently outperforms both traditional MIL and recent VLM-based MIL approaches, achieving gains of up to 4.1% in macro F1 under 16-shot settings. Our results demonstrate the value of jointly modeling hierarchical structure and multimodal alignment for efficient and scalable learning from limited pathology data. The code is available at https://github.com/bryanwong17/HiVE-MIL.

Method: Introduced Max-Former with two frequency-enhancing operators: (1) extra Max-Pool in patch embedding, and (2) Depth-Wise Convolution replacing self-attention. Also developed Max-ResNet-18 for convolution-based benchmarks.

Result: Max-Former achieved 82.39% top-1 accuracy on ImageNet with only 63.99M parameters, surpassing Spikformer by +7.58%. Max-ResNet-18 achieved SOTA: 97.17% on CIFAR-10 and 83.06% on CIFAR-100.

Conclusion: Frequency bias, not binary activations, is the main limitation in SNNs. The proposed frequency-enhancing approach effectively restores high-frequency signals and significantly improves SNN performance across different architectures.

Abstract: Spiking Neural Networks promise brain-inspired and energy-efficient computation by transmitting information through binary (0/1) spikes. Yet, their performance still lags behind that of artificial neural networks, often assumed to result from information loss caused by sparse and binary activations. In this work, we challenge this long-standing assumption and reveal a previously overlooked frequency bias: spiking neurons inherently suppress high-frequency components and preferentially propagate low-frequency information. This frequency-domain imbalance, we argue, is the root cause of degraded feature representation in SNNs. Empirically, on Spiking Transformers, adopting Avg-Pooling (low-pass) for token mixing lowers performance to 76.73% on Cifar-100, whereas replacing it with Max-Pool (high-pass) pushes the top-1 accuracy to 79.12%. Accordingly, we introduce Max-Former that restores high-frequency signals through two frequency-enhancing operators: (1) extra Max-Pool in patch embedding, and (2) Depth-Wise Convolution in place of self-attention. Notably, Max-Former attains 82.39% top-1 accuracy on ImageNet using only 63.99M parameters, surpassing Spikformer (74.81%, 66.34M) by +7.58%. Extending our insight beyond transformers, our Max-ResNet-18 achieves state-of-the-art performance on convolution-based benchmarks: 97.17% on CIFAR-10 and 83.06% on CIFAR-100. We hope this simple yet effective solution inspires future research to explore the distinctive nature of spiking neural networks. Code is available: https://github.com/bic-L/MaxFormer.

Method: Proposed SAMA model with versatile spatio-temporal context aggregator and Segment Anything Model, trained on SAMA-239K dataset (15K videos) for joint learning of video referring understanding, grounding, and multi-turn video chat.

Result: SAMA achieves strong performance on SAMA-Bench, sets new state-of-the-art on general grounding benchmarks, and maintains competitive performance on standard visual understanding benchmarks.

Conclusion: The unified approach enables comprehensive referentially grounded video interaction, addressing the bottleneck in fine-grained spatio-temporal video understanding through integrated dataset, model, and benchmark contributions.

Abstract: Achieving fine-grained spatio-temporal understanding in videos remains a major challenge for current Video Large Multimodal Models (Video LMMs). Addressing this challenge requires mastering two core capabilities: video referring understanding, which captures the semantics of video regions, and video grounding, which segments object regions based on natural language descriptions. However, most existing approaches tackle these tasks in isolation, limiting progress toward unified, referentially grounded video interaction. We identify a key bottleneck in the lack of high-quality, unified video instruction data and a comprehensive benchmark for evaluating referentially grounded video chat. To address these challenges, we contribute in three core aspects: dataset, model, and benchmark. First, we introduce SAMA-239K, a large-scale dataset comprising 15K videos specifically curated to enable joint learning of video referring understanding, grounding, and multi-turn video chat. Second, we propose the SAMA model, which incorporates a versatile spatio-temporal context aggregator and a Segment Anything Model to jointly enhance fine-grained video comprehension and precise grounding capabilities. Finally, we establish SAMA-Bench, a meticulously designed benchmark consisting of 5,067 questions from 522 videos, to comprehensively evaluate the integrated capabilities of Video LMMs in multi-turn, spatio-temporal referring understanding and grounded dialogue. Extensive experiments and benchmarking results show that SAMA not only achieves strong performance on SAMA-Bench but also sets a new state-of-the-art on general grounding benchmarks, while maintaining highly competitive performance on standard visual understanding benchmarks.

Method: The benchmark uses three question types: noncausal one-needle, causal one-needle, and causal two-needle questions. It introduces two question formats to prevent textual bias: locating video clips containing answers and verbal descriptions of visual details.

Result: Models that perform well on existing benchmarks struggle with causal two-needle questions, and performance decreases as the distance between the two information locations increases.

Conclusion: Current VLMs have critical limitations in understanding long videos, particularly in extracting information from multiple locations and modeling cause-effect relationships.

Abstract: Properly evaluating the ability of Video-Language Models (VLMs) to understand long videos remains a challenge. We propose a long-context video understanding benchmark, Causal2Needles, that assesses two crucial abilities insufficiently addressed by existing benchmarks: (1) extracting information from two separate locations (two needles) in a long video and understanding them jointly, and (2) modeling the world in terms of cause and effect in human behaviors. Causal2Needles evaluates these abilities using noncausal one-needle, causal one-needle, and causal two-needle questions. The most complex question type, causal two-needle questions, require extracting information from both the cause and effect events from a long video and the associated narration text. To prevent textual bias, we introduce two complementary question formats: locating the video clip containing the answer, and verbal description of a visual detail from that video clip. Our experiments reveal that models excelling on existing benchmarks struggle with causal 2-needle questions, and the model performance is negatively correlated with the distance between the two needles. These findings highlight critical limitations in current VLMs. The dataset is available at: https://huggingface.co/datasets/causal2needles/Causal2Needles

Method: Proposes an identity-preserving motion-controllable video Diffusion Transformer architecture and a semi-automatically curated dataset for the Frame In and Frame Out task.

Result: The evaluation shows that the proposed approach significantly outperforms existing baselines in controlling object entry/exit from scenes.

Conclusion: The method successfully enables controllable object motion in video generation using Frame In and Frame Out techniques with improved performance over existing approaches.

Abstract: Controllability, temporal coherence, and detail synthesis remain the most critical challenges in video generation. In this paper, we focus on a commonly used yet underexplored cinematic technique known as Frame In and Frame Out. Specifically, starting from image-to-video generation, users can control the objects in the image to naturally leave the scene or provide breaking new identity references to enter the scene, guided by a user-specified motion trajectory. To support this task, we introduce a new dataset that is curated semi-automatically, an efficient identity-preserving motion-controllable video Diffusion Transformer architecture, and a comprehensive evaluation protocol targeting this task. Our evaluation shows that our proposed approach significantly outperforms existing baselines.

Method: Proposed framework augments egocentric inputs with exocentric views, created E3VQA benchmark dataset with 4K question-answer pairs, and developed M3CoT prompting technique that integrates scene graphs from three complementary perspectives.

Result: M3CoT achieved consistent performance gains (4.84% for GPT-4o and 5.94% for Gemini 2.0 Flash) over recent chain-of-thought baselines, revealing key strengths and limitations of LVLMs in multi-view reasoning.

Conclusion: Combining egocentric and exocentric inputs provides valuable complementary information for comprehensive scene understanding in vision-language models.

Abstract: Large vision-language models (LVLMs) are increasingly deployed in interactive applications such as virtual and augmented reality, where a first-person (egocentric) view captured by head-mounted cameras serves as key input. While this view offers fine-grained cues about user attention and hand-object interactions, its narrow field of view and lack of global context often lead to failures on spatially or contextually demanding queries. To address this, we introduce a framework that augments egocentric inputs with third-person (exocentric) views, providing complementary information such as global scene layout and object visibility to LVLMs. We present E3VQA, the first benchmark for multi-view question answering with 4K high-quality question-answer pairs grounded in synchronized ego-exo image pairs. Additionally, we propose M3CoT, a training-free prompting technique that constructs a unified scene representation by integrating scene graphs from three complementary perspectives. M3CoT enables LVLMs to reason more effectively across views, yielding consistent performance gains (4.84% for GPT-4o and 5.94% for Gemini 2.0 Flash) over a recent CoT baseline. Our extensive evaluation reveals key strengths and limitations of LVLMs in multi-view reasoning and highlights the value of leveraging both egocentric and exocentric inputs. The dataset and source code are available at https://github.com/Leeinsu1/Towards-Comprehensive-Scene-Understanding.

Method: Alternative training paradigms combining hard-data-mining and dropout insights, progressively reducing data used during training.

Result: Achieves 12.4% of baseline training epochs with accuracy improvements up to 4.82%, no architecture modifications needed.

Conclusion: Progressive Data Dropout offers efficient training alternative that reduces computational costs while maintaining or improving accuracy, promising for wide adoption.

Abstract: The success of the machine learning field has reliably depended on training on large datasets. While effective, this trend comes at an extraordinary cost. This is due to two deeply intertwined factors: the size of models and the size of datasets. While promising research efforts focus on reducing the size of models, the other half of the equation remains fairly mysterious. Indeed, it is surprising that the standard approach to training remains to iterate over and over, uniformly sampling the training dataset. In this paper we explore a series of alternative training paradigms that leverage insights from hard-data-mining and dropout, simple enough to implement and use that can become the new training standard. The proposed Progressive Data Dropout reduces the number of effective epochs to as little as 12.4% of the baseline. This savings actually do not come at any cost for accuracy. Surprisingly, the proposed method improves accuracy by up to 4.82%. Our approach requires no changes to model architecture or optimizer, and can be applied across standard training pipelines, thus posing an excellent opportunity for wide adoption. Code can be found here: https://github.com/bazyagami/LearningWithRevision

Method: RiverMamba uses efficient Mamba blocks pretrained with long-term reanalysis data to capture spatio-temporal relations in large river networks. It integrates ECMWF HRES meteorological forecasts while accounting for their inaccuracies through spatio-temporal modeling on a 0.05° grid.

Result: RiverMamba provides reliable predictions of river discharge across various flood return periods, including extreme floods, and different lead times, surpassing both AI-based and physics-based models in performance.

Conclusion: The model successfully addresses the need for spatio-temporal modeling in hydrology and demonstrates superior forecasting capability for global river discharge and flood prediction, with publicly available source code and datasets.

Abstract: Recent deep learning approaches for river discharge forecasting have improved the accuracy and efficiency in flood forecasting, enabling more reliable early warning systems for risk management. Nevertheless, existing deep learning approaches in hydrology remain largely confined to local-scale applications and do not leverage the inherent spatial connections of bodies of water. Thus, there is a strong need for new deep learning methodologies that are capable of modeling spatio-temporal relations to improve river discharge and flood forecasting for scientific and operational applications. To address this, we present RiverMamba, a novel deep learning model that is pretrained with long-term reanalysis data and that can forecast global river discharge and floods on a $0.05^\circ$ grid up to $7$ days lead time, which is of high relevance in early warning. To achieve this, RiverMamba leverages efficient Mamba blocks that enable the model to capture spatio-temporal relations in very large river networks and enhance its forecast capability for longer lead times. The forecast blocks integrate ECMWF HRES meteorological forecasts, while accounting for their inaccuracies through spatio-temporal modeling. Our analysis demonstrates that RiverMamba provides reliable predictions of river discharge across various flood return periods, including extreme floods, and lead times, surpassing both AI- and physics-based models. The source code and datasets are publicly available at the project page https://hakamshams.github.io/RiverMamba.

Method: Operates directly on Gaussian primitives and integrates as a plug-in module into existing GS pipelines, enabling joint optimization of color and geometry in 3D/4D settings while maintaining temporal coherence in videos.

Result: Achieves superior style fidelity and consistency across all tasks while preserving model size, demonstrating effectiveness as a universal solution for multimodal style transfer.

Conclusion: CLIPGaussian provides an efficient and unified framework for multimodal style transfer that works across 2D, 3D, and 4D content without the need for large generative models or complete retraining.

Abstract: Gaussian Splatting (GS) has recently emerged as an efficient representation for rendering 3D scenes from 2D images and has been extended to images, videos, and dynamic 4D content. However, applying style transfer to GS-based representations, especially beyond simple color changes, remains challenging. In this work, we introduce CLIPGaussian, the first unified style transfer framework that supports text- and image-guided stylization across multiple modalities: 2D images, videos, 3D objects, and 4D scenes. Our method operates directly on Gaussian primitives and integrates into existing GS pipelines as a plug-in module, without requiring large generative models or retraining from scratch. The CLIPGaussian approach enables joint optimization of color and geometry in 3D and 4D settings, and achieves temporal coherence in videos, while preserving the model size. We demonstrate superior style fidelity and consistency across all tasks, validating CLIPGaussian as a universal and efficient solution for multimodal style transfer.

Method: Uses a shared feature adapter trained collaboratively across clients to adapt features from foundation models, and a shared slot attention module for reconstruction. Implements a two-branch student-teacher architecture where student decoders reconstruct full features and teacher decoders reconstruct adapted low-dimensional features.

Result: Outperforms centralized baselines on object discovery and learns a compact, universal representation that generalizes well across domains in multiple real-world datasets.

Conclusion: Federated slot attention is an effective tool for scalable, unsupervised visual representation learning from cross-domain data with distributed concepts.

Abstract: Learning efficient visual representations across heterogeneous unlabeled datasets remains a central challenge in federated learning. Effective federated representations require features that are jointly informative across clients while disentangling domain-specific factors without supervision. We introduce FORLA, a novel framework for federated object-centric representation learning and feature adaptation across clients using unsupervised slot attention. At the core of our method is a shared feature adapter, trained collaboratively across clients to adapt features from foundation models, and a shared slot attention module that learns to reconstruct the adapted features. To optimize this adapter, we design a two-branch student-teacher architecture. In each client, a student decoder learns to reconstruct full features from foundation models, while a teacher decoder reconstructs their adapted, low-dimensional counterpart. The shared slot attention module bridges cross-domain learning by aligning object-level representations across clients. Experiments in multiple real-world datasets show that our framework not only outperforms centralized baselines on object discovery but also learns a compact, universal representation that generalizes well across domains. This work highlights federated slot attention as an effective tool for scalable, unsupervised visual representation learning from cross-domain data with distributed concepts.

Method: ArrowRL uses reinforcement learning with an innovative reverse reward that instills Arrow of Time awareness by encouraging divergent video interpretations between forward and reversed visual frames.

Result: ArrowRL achieves substantial improvements on the challenging AoTBench benchmark and boosts performance on standard video question answering benchmarks, with peak accuracy gains reaching over 20% and 10% respectively.

Conclusion: The approach validates ArrowRL’s effectiveness and highlights the critical need for dedicated Arrow of Time understanding in large multimodal models for better video comprehension.

Abstract: The Arrow of Time (AoT)-time’s irreversible flow shaping physical events-is fundamental to video comprehension, yet remains a significant challenge for modern large multimodal models (LMMs). Current LMMs struggle to perceive and utilize temporal directionality in video when responding to language queries, obstructing deeper temporal understanding. We tackle this deficiency by first providing a critical analysis of existing benchmarks and models. We then introduce ArrowRL, a reinforcement learning (RL)-based training strategy with an innovative reverse reward that instills AoT awareness by encouraging divergent video interpretations between forward and reversed visual frames. For rigorous evaluation, we additionally develop AoTBench, a new multi-faceted benchmark probing temporally challenging questions. Experiments show ArrowRL greatly advances temporal perception: it not only achieves substantial improvements on our challenging AoTBench but also demonstrably boosts performance on standard video question answering (VQA) benchmarks (with peak accuracy gains reaching over 20% and 10% respectively). This validates ArrowRL’s effectiveness and highlights the critical need for dedicated AoT understanding in LMMs.

Method: 1) Constructs skill-based CoT annotations by extracting domain-relevant reasoning skills, clustering them into a shared taxonomy, and creating tailored multi-step rationales. 2) Uses skill-specific expert learning with lightweight adapters trained on the collected CoT supervision.

Result: Video-SKoT consistently outperforms strong baselines on three video understanding benchmarks and provides in-depth analyses of different CoT annotation pipelines and learned skills.

Conclusion: The proposed framework effectively addresses domain adaptation challenges in video reasoning through automated skill-aware CoT supervision and specialized expert learning.

Abstract: Recent advances in Chain-of-Thought (CoT) reasoning have improved complex video understanding, but existing methods often struggle to adapt to domain-specific skills (e.g., event detection, spatial relation understanding, emotion understanding) over various video content. To address this, we propose Video-Skill-CoT (a.k.a. Video-SKoT), a framework that automatically constructs and leverages skill-aware CoT supervisions for domain-adaptive video reasoning. First, we construct skill-based CoT annotations: we extract domain-relevant reasoning skills from training questions, cluster them into a shared skill taxonomy, and create detailed multi-step CoT rationale tailored to each video-question pair for training. Second, we introduce a skill-specific expert learning framework. Each expert module specializes in a subset of reasoning skills and is trained with lightweight adapters using the collected CoT supervision. We demonstrate the effectiveness of the proposed approach on three video understanding benchmarks, where Video-SKoT consistently outperforms strong baselines. We also provide in-depth analyses on comparing different CoT annotation pipelines and learned skills over multiple video domains.

Method: Learns a continuous point-wise velocity field that transports noisy points toward target positions, recovering part poses without symmetry labels, combined with a self-supervised encoder focused on overlapping points.

Result: Achieves state-of-the-art performance on six benchmarks for pairwise registration and shape assembly, with joint training enabling shared geometric priors that boost accuracy.

Conclusion: The unified formulation effectively handles both registration and assembly tasks, intrinsically learns symmetries without labels, and enables joint training for improved performance through shared geometric knowledge.

Abstract: We introduce Rectified Point Flow, a unified parameterization that formulates pairwise point cloud registration and multi-part shape assembly as a single conditional generative problem. Given unposed point clouds, our method learns a continuous point-wise velocity field that transports noisy points toward their target positions, from which part poses are recovered. In contrast to prior work that regresses part-wise poses with ad-hoc symmetry handling, our method intrinsically learns assembly symmetries without symmetry labels. Together with a self-supervised encoder focused on overlapping points, our method achieves a new state-of-the-art performance on six benchmarks spanning pairwise registration and shape assembly. Notably, our unified formulation enables effective joint training on diverse datasets, facilitating the learning of shared geometric priors and consequently boosting accuracy. Project page: https://rectified-pointflow.github.io/.

Method: Uses Foreground-Aware Pseudo-Anomalies in a Vision Transformer architecture with adversarial training guided by a novel loss function derived from theoretical analysis of attention mechanisms.

Result: Significantly outperforms previous methods in adversarial settings with 53.2% improvement in anomaly detection and 68.5% improvement in anomaly localization, while maintaining competitive accuracy in non-adversarial settings.

Conclusion: PatchGuard provides effective adversarial robustness for anomaly detection and localization systems, addressing critical vulnerabilities in high-reliability applications through pseudo-anomaly integration and ViT-based architecture.

Abstract: Anomaly Detection (AD) and Anomaly Localization (AL) are crucial in fields that demand high reliability, such as medical imaging and industrial monitoring. However, current AD and AL approaches are often susceptible to adversarial attacks due to limitations in training data, which typically include only normal, unlabeled samples. This study introduces PatchGuard, an adversarially robust AD and AL method that incorporates pseudo anomalies with localization masks within a Vision Transformer (ViT)-based architecture to address these vulnerabilities. We begin by examining the essential properties of pseudo anomalies, and follow it by providing theoretical insights into the attention mechanisms required to enhance the adversarial robustness of AD and AL systems. We then present our approach, which leverages Foreground-Aware Pseudo-Anomalies to overcome the deficiencies of previous anomaly-aware methods. Our method incorporates these crafted pseudo-anomaly samples into a ViT-based framework, with adversarial training guided by a novel loss function designed to improve model robustness, as supported by our theoretical analysis. Experimental results on well-established industrial and medical datasets demonstrate that PatchGuard significantly outperforms previous methods in adversarial settings, achieving performance gains of $53.2%$ in AD and $68.5%$ in AL, while also maintaining competitive accuracy in non-adversarial settings. The code repository is available at https://github.com/rohban-lab/PatchGuard .

Method: Angle-Robust Concept Learning (AngleRoCL) learns generalizable text embeddings that represent the capability of generating angle-robust patches. These learned concepts are incorporated into textual prompts to guide T2I models in generating patches inherently resistant to viewpoint variations.

Result: Extensive experiments on five state-of-the-art detectors across multiple views show that AngleRoCL significantly enhances angle robustness, with over 50% average relative improvement in attack effectiveness across multiple angles. The patches maintain high attack success rates even under challenging viewing conditions.

Conclusion: AngleRoCL advances the understanding of physically angle-robust patches and provides insights into the relationship between textual concepts and physical properties in T2I-generated content. The method effectively addresses the angle robustness issue in adversarial patch generation.

Abstract: Cutting-edge works have demonstrated that text-to-image (T2I) diffusion models can generate adversarial patches that mislead state-of-the-art object detectors in the physical world, revealing detectors’ vulnerabilities and risks. However, these methods neglect the T2I patches’ attack effectiveness when observed from different views in the physical world (i.e., angle robustness of the T2I adversarial patches). In this paper, we study the angle robustness of T2I adversarial patches comprehensively, revealing their angle-robust issues, demonstrating that texts affect the angle robustness of generated patches significantly, and task-specific linguistic instructions fail to enhance the angle robustness. Motivated by the studies, we introduce Angle-Robust Concept Learning (AngleRoCL), a simple and flexible approach that learns a generalizable concept (i.e., text embeddings in implementation) representing the capability of generating angle-robust patches. The learned concept can be incorporated into textual prompts and guides T2I models to generate patches with their attack effectiveness inherently resistant to viewpoint variations. Through extensive simulation and physical-world experiments on five SOTA detectors across multiple views, we demonstrate that AngleRoCL significantly enhances the angle robustness of T2I adversarial patches compared to baseline methods. Our patches maintain high attack success rates even under challenging viewing conditions, with over 50% average relative improvement in attack effectiveness across multiple angles. This research advances the understanding of physically angle-robust patches and provides insights into the relationship between textual concepts and physical properties in T2I-generated contents. We released our code at https://github.com/tsingqguo/anglerocl.

Method: Mixes class-conditioned scores along reverse diffusion trajectories, systematically studying class-selection strategies to find that mixing distant classes in discriminator’s embedding space yields better results.

Result: Improves accuracy by up to 7 percentage points across 8 public face recognition benchmarks without hyperparameter search, with distant class mixing providing up to 3% additional improvement.

Conclusion: ScoreMix provides a simple yet effective way to maximize discriminator performance using only available datasets, without reliance on third-party resources.

Abstract: Synthetic data generation is increasingly used in machine learning for training and data augmentation. Yet, current strategies often rely on external foundation models or datasets, whose usage is restricted in many scenarios due to policy or legal constraints. We propose ScoreMix, a self-contained synthetic generation method to produce hard synthetic samples for recognition tasks by leveraging the score compositionality of diffusion models. The approach mixes class-conditioned scores along reverse diffusion trajectories, yielding domain-specific data augmentation without external resources. We systematically study class-selection strategies and find that mixing classes distant in the discriminator’s embedding space yields larger gains, providing up to 3% additional average improvement, compared to selection based on proximity. Interestingly, we observe that condition and embedding spaces are largely uncorrelated under standard alignment metrics, and the generator’s condition space has a negligible effect on downstream performance. Across 8 public face recognition benchmarks, ScoreMix improves accuracy by up to 7 percentage points, without hyperparameter search, highlighting both robustness and practicality. Our method provides a simple yet effective way to maximize discriminator performance using only the available dataset, without reliance on third-party resources. Paper website: https://parsa-ra.github.io/scoremix/.

Method: Reformulates densification and pruning as probabilistic sampling using Metropolis-Hastings approach with Bayesian acceptance tests based on aggregated multi-view errors and opacity scores.

Result: Reduces number of Gaussians needed, achieves faster convergence while matching or modestly surpassing view-synthesis quality on Mip-NeRF360, Tanks and Temples, and Deep Blending datasets.

Conclusion: The framework substantially reduces reliance on heuristics, offers greater flexibility, and adaptively infers Gaussian distributions without requiring predefined scene complexity.

Abstract: We propose an adaptive sampling framework for 3D Gaussian Splatting (3DGS) that leverages comprehensive multi-view photometric error signals within a unified Metropolis-Hastings approach. Vanilla 3DGS heavily relies on heuristic-based density-control mechanisms (e.g., cloning, splitting, and pruning), which can lead to redundant computations or premature removal of beneficial Gaussians. Our framework overcomes these limitations by reformulating densification and pruning as a probabilistic sampling process, dynamically inserting and relocating Gaussians based on aggregated multi-view errors and opacity scores. Guided by Bayesian acceptance tests derived from these error-based importance scores, our method substantially reduces reliance on heuristics, offers greater flexibility, and adaptively infers Gaussian distributions without requiring predefined scene complexity. Experiments on benchmark datasets, including Mip-NeRF360, Tanks and Temples and Deep Blending, show that our approach reduces the number of Gaussians needed, achieving faster convergence while matching or modestly surpassing the view-synthesis quality of state-of-the-art models.

Method: Constructs MP16-Reason dataset using diverse social media images. Introduces GLOBE with group-relative policy optimization for localizability assessment and optimized visual-cue reasoning, incorporating task-specific rewards for joint enhancement of localizability, reasoning, and geolocation accuracy.

Result: Outperforms state-of-the-art open-source LVLMs on geo-localization tasks, especially in diverse visual scenes, and generates more insightful and interpretable reasoning trajectories.

Conclusion: GLOBE successfully addresses data and modeling challenges in geo-localization by creating diverse reasoning datasets and enhancing VLM reasoning capabilities through multi-objective optimization, achieving superior performance and interpretability.

Abstract: Previous methods for image geo-localization have typically treated the task as either classification or retrieval, often relying on black-box decisions that lack interpretability. The rise of large vision-language models (LVLMs) has enabled a rethinking of geo-localization as a reasoning-driven task grounded in visual cues. However, two major challenges persist. On the data side, existing reasoning-focused datasets are primarily based on street-view imagery, offering limited scene diversity and constrained viewpoints. On the modeling side, current approaches predominantly rely on supervised fine-tuning, which yields only marginal improvements in reasoning capabilities. To address these challenges, we propose a novel pipeline that constructs a reasoning-oriented geo-localization dataset, MP16-Reason, using diverse social media images. We introduce GLOBE, Group-relative policy optimization for Localizability assessment and Optimized visual-cue reasoning, yielding Bi-objective geo-Enhancement for the VLM in recognition and reasoning. GLOBE incorporates task-specific rewards that jointly enhance localizability assessment, visual-cue reasoning, and geolocation accuracy. Both qualitative and quantitative results demonstrate that GLOBE outperforms state-of-the-art open-source LVLMs on geo-localization tasks, particularly in diverse visual scenes, while also generating more insightful and interpretable reasoning trajectories. The data and code are available at https://github.com/lingli1996/GLOBE.

Method: Created a data collection platform with two vehicles (each with 5 cameras and 1 LiDAR) and one UAV (with forward-facing camera and LiDAR), collecting approximately 80K LiDAR frames and 360K images across 14 diverse driving scenarios including 17% dynamic interaction events.

Result: Produced AGC-Drive dataset with 350 scenes, each with ~100 frames and fully annotated 3D bounding boxes covering 13 object categories, providing benchmarks for vehicle-to-vehicle and vehicle-to-UAV collaborative perception.

Conclusion: AGC-Drive enables research in aerial-ground cooperative 3D perception and comes with open-source toolkit including alignment verification, visualization systems, and annotation utilities.

Abstract: By sharing information across multiple agents, collaborative perception helps autonomous vehicles mitigate occlusions and improve overall perception accuracy. While most previous work focus on vehicle-to-vehicle and vehicle-to-infrastructure collaboration, with limited attention to aerial perspectives provided by UAVs, which uniquely offer dynamic, top-down views to alleviate occlusions and monitor large-scale interactive environments. A major reason for this is the lack of high-quality datasets for aerial-ground collaborative scenarios. To bridge this gap, we present AGC-Drive, the first large-scale real-world dataset for Aerial-Ground Cooperative 3D perception. The data collection platform consists of two vehicles, each equipped with five cameras and one LiDAR sensor, and one UAV carrying a forward-facing camera and a LiDAR sensor, enabling comprehensive multi-view and multi-agent perception. Consisting of approximately 80K LiDAR frames and 360K images, the dataset covers 14 diverse real-world driving scenarios, including urban roundabouts, highway tunnels, and on/off ramps. Notably, 17% of the data comprises dynamic interaction events, including vehicle cut-ins, cut-outs, and frequent lane changes. AGC-Drive contains 350 scenes, each with approximately 100 frames and fully annotated 3D bounding boxes covering 13 object categories. We provide benchmarks for two 3D perception tasks: vehicle-to-vehicle collaborative perception and vehicle-to-UAV collaborative perception. Additionally, we release an open-source toolkit, including spatiotemporal alignment verification tools, multi-agent visualization systems, and collaborative annotation utilities. The dataset and code are available at https://github.com/PercepX/AGC-Drive.

Method: Proposes IPFormer which adaptively initializes queries as panoptic instance proposals from image context and refines them through attention-based encoding and decoding to reason about semantic instance-voxel relationships.

Result: Achieves state-of-the-art in-domain performance, superior zero-shot generalization on out-of-domain data, and over 14x runtime reduction compared to existing methods.

Conclusion: The introduction of context-adaptive instance proposals represents a pioneering approach for vision-based 3D Panoptic Scene Completion, demonstrating significant improvements in performance, generalization, and efficiency.

Abstract: Semantic Scene Completion (SSC) has emerged as a pivotal approach for jointly learning scene geometry and semantics, enabling downstream applications such as navigation in mobile robotics. The recent generalization to Panoptic Scene Completion (PSC) advances the SSC domain by integrating instance-level information, thereby enhancing object-level sensitivity in scene understanding. While PSC was introduced using LiDAR modality, methods based on camera images remain largely unexplored. Moreover, recent Transformer-based approaches utilize a fixed set of learned queries to reconstruct objects within the scene volume. Although these queries are typically updated with image context during training, they remain static at test time, limiting their ability to dynamically adapt specifically to the observed scene. To overcome these limitations, we propose IPFormer, the first method that leverages context-adaptive instance proposals at train and test time to address vision-based 3D Panoptic Scene Completion. Specifically, IPFormer adaptively initializes these queries as panoptic instance proposals derived from image context and further refines them through attention-based encoding and decoding to reason about semantic instance-voxel relationships. Extensive experimental results show that our approach achieves state-of-the-art in-domain performance, exhibits superior zero-shot generalization on out-of-domain data, and achieves a runtime reduction exceeding 14x. These results highlight our introduction of context-adaptive instance proposals as a pioneering effort in addressing vision-based 3D Panoptic Scene Completion.

Method: Created a benchmark with curated text prompts and human-selected pose conditioning images, proposed multi-faceted evaluation using four metrics, and introduced techniques using human segmentation and Hungarian matching for improved ID similarity.

Result: The benchmark enables thorough evaluation of diverse models including zero-shot and training-based methods, with proposed techniques significantly improving ID similarity.

Conclusion: MultiHuman-Testbench provides a standardized tool and valuable insights for advancing research in multi-human image generation.

Abstract: Generation of images containing multiple humans, performing complex actions, while preserving their facial identities, is a significant challenge. A major factor contributing to this is the lack of a dedicated benchmark. To address this, we introduce MultiHuman-Testbench, a novel benchmark for rigorously evaluating generative models for multi-human generation. The benchmark comprises 1,800 samples, including carefully curated text prompts, describing a range of simple to complex human actions. These prompts are matched with a total of 5,550 unique human face images, sampled uniformly to ensure diversity across age, ethnic background, and gender. Alongside captions, we provide human-selected pose conditioning images which accurately match the prompt. We propose a multi-faceted evaluation suite employing four key metrics to quantify face count, ID similarity, prompt alignment, and action detection. We conduct a thorough evaluation of a diverse set of models, including zero-shot approaches and training-based methods, with and without regional priors. We also propose novel techniques to incorporate image and region isolation using human segmentation and Hungarian matching, significantly improving ID similarity. Our proposed benchmark and key findings provide valuable insights and a standardized tool for advancing research in multi-human image generation. The dataset and evaluation codes will be available at https://github.com/Qualcomm-AI-research/MultiHuman-Testbench.

Method: Combines data-efficient RL with video-adaptive test-time scaling strategy, skipping resource-intensive SFT step and using pure-RL training with output-based rewards, plus sparse-to-dense video TTS that iteratively adds frames based on output consistency.

Result: Surpasses existing video reasoning models by 2.4% in accuracy using only 3.6% training samples, with 4.2% improvement on Video-Holmes benchmark.

Conclusion: Pure RL training and adaptive video TTS offer complementary strengths, enabling Video-RTS’s strong reasoning performance with drastically improved data efficiency.

Abstract: Despite advances in reinforcement learning (RL)-based video reasoning with large language models (LLMs), data collection and fine-tuning remain significant challenges. These methods often rely on large-scale supervised fine-tuning (SFT) with extensive video data and long Chain-of-Thought (CoT) annotations, making them costly and hard to scale. To address this, we present Video-RTS, a new approach to improve video reasoning capability with drastically improved data efficiency by combining data-efficient RL with a video-adaptive test-time scaling (TTS) strategy. Building on observations about the data scaling, we skip the resource-intensive SFT step and employ efficient pure-RL training with output-based rewards, requiring no additional annotations or extensive fine-tuning. Furthermore, to utilize computational resources more efficiently, we introduce a sparse-to-dense video TTS strategy that improves inference by iteratively adding frames based on output consistency. We validate our approach on multiple video reasoning benchmarks, showing that Video-RTS surpasses existing video reasoning models by 2.4% in accuracy using only 3.6% training samples. Specifically, Video-RTS achieves a 4.2% improvement on Video-Holmes, a recent and challenging video reasoning benchmark. Notably, our pure RL training and adaptive video TTS offer complementary strengths, enabling Video-RTS’s strong reasoning performance.

Method: Constructed a tailor-made tangential Raviart-Thomas type finite element space to handle the manifold setting, extending previous Raviart-Thomas approaches for piecewise constant scalar data.

Result: The new regularizer was evaluated in mesh denoising experiments and compared to existing methods.

Conclusion: The proposed formulation successfully extends TGV to manifold-valued normal vector functions, providing a new approach for mesh denoising applications.

Abstract: We propose a novel formulation for the second-order total generalized variation (TGV) of the normal vector on an oriented, triangular mesh embedded in $\R^3$. The normal vector is considered as a manifold-valued function, taking values on the unit sphere. Our formulation extends previous discrete TGV models for piecewise constant scalar data that utilize a Raviart-Thomas function space. To extend this formulation to the manifold setting, a tailor-made tangential Raviart-Thomas type finite element space is constructed in this work. The new regularizer is compared to existing methods in mesh denoising experiments.

Method: Comprehensive evaluation of pre-trained DUSt3R/MASt3R/VGGT models on aerial blocks from the UseGeo dataset for pose estimation and dense 3D reconstruction, testing with very sparse image sets (fewer than 10 images) at up to 518 pixels resolution.

Result: Methods accurately reconstruct dense point clouds from very sparse image sets with completeness gains up to +50% over COLMAP. VGGT shows higher computational efficiency, scalability, and more reliable camera pose estimation. However, all models exhibit limitations with high-resolution images and large sets, with pose reliability declining as image count and geometric complexity increase.

Conclusion: Transformer-based methods cannot fully replace traditional SfM and MVS, but offer promise as complementary approaches, especially in challenging, low-resolution, and sparse scenarios where they outperform traditional methods.

Abstract: State-of-the-art 3D computer vision algorithms continue to advance in handling sparse, unordered image sets. Recently developed foundational models for 3D reconstruction, such as Dense and Unconstrained Stereo 3D Reconstruction (DUSt3R), Matching and Stereo 3D Reconstruction (MASt3R), and Visual Geometry Grounded Transformer (VGGT), have attracted attention due to their ability to handle very sparse image overlaps. Evaluating DUSt3R/MASt3R/VGGT on typical aerial images matters, as these models may handle extremely low image overlaps, stereo occlusions, and textureless regions. For redundant collections, they can accelerate 3D reconstruction by using extremely sparsified image sets. Despite tests on various computer vision benchmarks, their potential on photogrammetric aerial blocks remains unexplored. This paper conducts a comprehensive evaluation of the pre-trained DUSt3R/MASt3R/VGGT models on the aerial blocks of the UseGeo dataset for pose estimation and dense 3D reconstruction. Results show these methods can accurately reconstruct dense point clouds from very sparse image sets (fewer than 10 images, up to 518 pixels resolution), with completeness gains up to +50% over COLMAP. VGGT also demonstrates higher computational efficiency, scalability, and more reliable camera pose estimation. However, all exhibit limitations with high-resolution images and large sets, as pose reliability declines with more images and geometric complexity. These findings suggest transformer-based methods cannot fully replace traditional SfM and MVS, but offer promise as complementary approaches, especially in challenging, low-resolution, and sparse scenarios.

Method: Proposes DC3 with Color Compensation: after calibrated selection, uses latent diffusion model to enhance color diversity of images rather than creating new ones, and fine-tunes pre-trained diffusion models with condensed datasets.

Result: Superior performance and generalization across multiple benchmarks; achieves high-quality datasets without model collapse or degradation issues as proven by FID and Inception Score results.

Conclusion: DC3 successfully addresses the trade-off between performance and fidelity in dataset condensation by focusing on color compensation, making training networks with condensed datasets feasible without quality degradation.

Abstract: Dataset condensation always faces a constitutive trade-off: balancing performance and fidelity under extreme compression. Existing methods struggle with two bottlenecks: image-level selection methods (Coreset Selection, Dataset Quantization) suffer from inefficiency condensation, while pixel-level optimization (Dataset Distillation) introduces semantic distortion due to over-parameterization. With empirical observations, we find that a critical problem in dataset condensation is the oversight of color’s dual role as an information carrier and a basic semantic representation unit. We argue that improving the colorfulness of condensed images is beneficial for representation learning. Motivated by this, we propose DC3: a Dataset Condensation framework with Color Compensation. After a calibrated selection strategy, DC3 utilizes the latent diffusion model to enhance the color diversity of an image rather than creating a brand-new one. Extensive experiments demonstrate the superior performance and generalization of DC3 that outperforms SOTA methods across multiple benchmarks. To the best of our knowledge, besides focusing on downstream tasks, DC3 is the first research to fine-tune pre-trained diffusion models with condensed datasets. The Frechet Inception Distance (FID) and Inception Score (IS) results prove that training networks with our high-quality datasets is feasible without model collapse or other degradation issues. Code and generated data are available at https://github.com/528why/Dataset-Condensation-with-Color-Compensation.

Method: Integrates MVS and MDE features for better reconstruction, proposes projection-and-sampling mechanism for deep depth fusion with probability volume, introduces reference-view loss for geometry improvement, and uses 3D Gaussian representations for acceleration.

Result: Achieves state-of-the-art performance across multiple baseline settings and diverse scenarios (DTU, RealEstate10K), and shows promising zero-shot performance on LLFF and Mip-NeRF 360 datasets.

Conclusion: MuGS provides an effective generalized approach for novel view synthesis that handles diverse baseline settings efficiently while maintaining high rendering quality.

Abstract: We present Multi-Baseline Gaussian Splatting (MuGS), a generalized feed-forward approach for novel view synthesis that effectively handles diverse baseline settings, including sparse input views with both small and large baselines. Specifically, we integrate features from Multi-View Stereo (MVS) and Monocular Depth Estimation (MDE) to enhance feature representations for generalizable reconstruction. Next, We propose a projection-and-sampling mechanism for deep depth fusion, which constructs a fine probability volume to guide the regression of the feature map. Furthermore, We introduce a reference-view loss to improve geometry and optimization efficiency. We leverage 3D Gaussian representations to accelerate training and inference time while enhancing rendering quality. MuGS achieves state-of-the-art performance across multiple baseline settings and diverse scenarios ranging from simple objects (DTU) to complex indoor and outdoor scenes (RealEstate10K). We also demonstrate promising zero-shot performance on the LLFF and Mip-NeRF 360 datasets. Code is available at https://github.com/EuclidLou/MuGS.

Method: Proposed Text-Conditioned State Space Model (TCSSM) that dynamically predicts input-dependent parameters using both bi-temporal images and geo-disaster-related descriptions to extract domain-invariant features and align visual and textual data.

Result: Extensive experiments show superior performance compared to state-of-the-art models, demonstrating consistent improvement in handling domain shifts.

Conclusion: The TCSSM framework effectively addresses domain shift in CDVQA by leveraging multi-modal information, and the BrightVQA dataset facilitates future domain generalization research in this area.

Abstract: The Earth’s surface is constantly changing, and detecting these changes provides valuable insights that benefit various aspects of human society. While traditional change detection methods have been employed to detect changes from bi-temporal images, these approaches typically require expert knowledge for accurate interpretation. To enable broader and more flexible access to change information by non-expert users, the task of Change Detection Visual Question Answering (CDVQA) has been introduced. However, existing CDVQA methods have been developed under the assumption that training and testing datasets share similar distributions. This assumption does not hold in real-world applications, where domain shifts often occur. In this paper, the CDVQA task is revisited with a focus on addressing domain shift. To this end, a new multi-modal and multi-domain dataset, BrightVQA, is introduced to facilitate domain generalization research in CDVQA. Furthermore, a novel state space model, termed Text-Conditioned State Space Model (TCSSM), is proposed. The TCSSM framework is designed to leverage both bi-temporal imagery and geo-disaster-related textual information in an unified manner to extract domain-invariant features across domains. Input-dependent parameters existing in TCSSM are dynamically predicted by using both bi-temporal images and geo-disaster-related description, thereby facilitating the alignment between bi-temporal visual data and the associated textual descriptions. Extensive experiments are conducted to evaluate the proposed method against state-of-the-art models, and superior performance is consistently demonstrated. The code and dataset will be made publicly available upon acceptance at https://github.com/Elman295/TCSSM.

Method: Proposes DeltaFlow with a Δ scheme for efficient temporal feature extraction, Category-Balanced Loss for underrepresented classes, and Instance Consistency Loss for coherent object motion.

Result: Achieves state-of-the-art performance on Argoverse 2, Waymo and nuScenes datasets with up to 22% lower error and 2× faster inference compared to next-best multi-frame supervised method, plus strong cross-domain generalization.

Conclusion: DeltaFlow effectively leverages temporal information with minimal computational cost while addressing class imbalance and motion inconsistency issues, demonstrating superior performance and efficiency in scene flow estimation.

Abstract: Previous dominant methods for scene flow estimation focus mainly on input from two consecutive frames, neglecting valuable information in the temporal domain. While recent trends shift towards multi-frame reasoning, they suffer from rapidly escalating computational costs as the number of frames grows. To leverage temporal information more efficiently, we propose DeltaFlow ($\Delta$Flow), a lightweight 3D framework that captures motion cues via a $\Delta$ scheme, extracting temporal features with minimal computational cost, regardless of the number of frames. Additionally, scene flow estimation faces challenges such as imbalanced object class distributions and motion inconsistency. To tackle these issues, we introduce a Category-Balanced Loss to enhance learning across underrepresented classes and an Instance Consistency Loss to enforce coherent object motion, improving flow accuracy. Extensive evaluations on the Argoverse 2, Waymo and nuScenes datasets show that $\Delta$Flow achieves state-of-the-art performance with up to 22% lower error and $2\times$ faster inference compared to the next-best multi-frame supervised method, while also demonstrating a strong cross-domain generalization ability. The code is open-sourced at https://github.com/Kin-Zhang/DeltaFlow along with trained model weights.

Method: Introduces a Dynamic Skip Connection block with two components: Test-Time Training module for dynamic adaptation during inference, and Dynamic Multi-Scale Kernel module for adaptive kernel size selection based on global context.

Result: The DSC block demonstrates plug-and-play effectiveness across various U-like network architectures including CNN-based, Transformer-based, hybrid CNN-Transformer, and Mamba-based networks.

Conclusion: The proposed DSC block fundamentally enhances cross-layer connectivity through adaptive mechanisms and can be seamlessly integrated into existing U-like network structures.

Abstract: U-like networks have become fundamental frameworks in medical image segmentation through skip connections that bridge high-level semantics and low-level spatial details. Despite their success, conventional skip connections exhibit two key limitations: inter-feature constraints and intra-feature constraints. The inter-feature constraint refers to the static nature of feature fusion in traditional skip connections, where information is transmitted along fixed pathways regardless of feature content. The intra-feature constraint arises from the insufficient modeling of multi-scale feature interactions, thereby hindering the effective aggregation of global contextual information. To overcome these limitations, we propose a novel Dynamic Skip Connection (DSC) block that fundamentally enhances cross-layer connectivity through adaptive mechanisms. The DSC block integrates two complementary components. (1) Test-Time Training (TTT) module. This module addresses the inter-feature constraint by enabling dynamic adaptation of hidden representations during inference, facilitating content-aware feature refinement. (2) Dynamic Multi-Scale Kernel (DMSK) module. To mitigate the intra-feature constraint, this module adaptively selects kernel sizes based on global contextual cues, enhancing the network capacity for multi-scale feature integration. The DSC block is architecture-agnostic and can be seamlessly incorporated into existing U-like network structures. Extensive experiments demonstrate the plug-and-play effectiveness of the proposed DSC block across CNN-based, Transformer-based, hybrid CNN-Transformer, and Mamba-based U-like networks.

Method: Built on geometric priors from monocular depth estimations, uses SfM points anchored algorithm for semantic region extraction and alignment, and introduces geometry-guided supervision at virtual views with pixel and feature level constraints for 3D consistency.

Result: Achieves photorealistic renderings under challenging sparse-view and multi-appearance conditions, significantly outperforming existing approaches across different datasets.

Conclusion: MS-GS effectively addresses sparse-view reconstruction challenges with multi-appearance capabilities, demonstrating superior performance through geometric priors and multi-view constraints.

Abstract: In-the-wild photo collections often contain limited volumes of imagery and exhibit multiple appearances, e.g., taken at different times of day or seasons, posing significant challenges to scene reconstruction and novel view synthesis. Although recent adaptations of Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) have improved in these areas, they tend to oversmooth and are prone to overfitting. In this paper, we present MS-GS, a novel framework designed with Multi-appearance capabilities in Sparse-view scenarios using 3DGS. To address the lack of support due to sparse initializations, our approach is built on the geometric priors elicited from monocular depth estimations. The key lies in extracting and utilizing local semantic regions with a Structure-from-Motion (SfM) points anchored algorithm for reliable alignment and geometry cues. Then, to introduce multi-view constraints, we propose a series of geometry-guided supervision steps at virtual views in pixel and feature levels to encourage 3D consistency and reduce overfitting. We also introduce a dataset and an in-the-wild experiment setting to set up more realistic benchmarks. We demonstrate that MS-GS achieves photorealistic renderings under various challenging sparse-view and multi-appearance conditions, and outperforms existing approaches significantly across different datasets.

Method: Generate synthetic captions using multimodal large language models guided by Wikipedia-derived visual information and taxon-tailored format examples, then train BioCAP (BioCLIP with Captions) using these captions.

Result: BioCAP captures rich semantics and achieves strong performance in species classification and text-image retrieval, demonstrating the value of descriptive captions beyond labels.

Conclusion: Descriptive captions are valuable for bridging biological images with multimodal foundation models, and synthetic caption generation with domain-specific contexts can effectively address the scalability challenge.

Abstract: This work investigates descriptive captions as an additional source of supervision for biological multimodal foundation models. Images and captions can be viewed as complementary samples from the latent morphospace of a species, each capturing certain biological traits. Incorporating captions during training encourages alignment with this shared latent structure, emphasizing potentially diagnostic characters while suppressing spurious correlations. The main challenge, however, lies in obtaining faithful, instance-specific captions at scale. This requirement has limited the utilization of natural language supervision in organismal biology compared with many other scientific domains. We complement this gap by generating synthetic captions with multimodal large language models (MLLMs), guided by Wikipedia-derived visual information and taxon-tailored format examples. These domain-specific contexts help reduce hallucination and yield accurate, instance-based descriptive captions. Using these captions, we train BioCAP (i.e., BioCLIP with Captions), a biological foundation model that captures rich semantics and achieves strong performance in species classification and text-image retrieval. These results demonstrate the value of descriptive captions beyond labels in bridging biological images with multimodal foundation models.

Method: The BTL framework decomposes interactions into three phases: Blink (rapid detection/attention), Think (reasoning/decision-making), and Link (executable command generation). It includes automated blink data generation and a rule-based BTL Reward mechanism for reinforcement learning.

Result: The developed BTL-UI agent demonstrates competitive performance in both static GUI understanding and dynamic interaction tasks across comprehensive benchmarks.

Conclusion: The framework provides effective empirical validation for developing advanced GUI agents that better mimic natural human interaction patterns.

Abstract: In the field of AI-driven human-GUI interaction automation, while rapid advances in multimodal large language models and reinforcement fine-tuning techniques have yielded remarkable progress, a fundamental challenge persists: their interaction logic significantly deviates from natural human-GUI communication patterns. To fill this gap, we propose “Blink-Think-Link” (BTL), a brain-inspired framework for human-GUI interaction that mimics the human cognitive process between users and graphical interfaces. The system decomposes interactions into three biologically plausible phases: (1) Blink - rapid detection and attention to relevant screen areas, analogous to saccadic eye movements; (2) Think - higher-level reasoning and decision-making, mirroring cognitive planning; and (3) Link - generation of executable commands for precise motor control, emulating human action selection mechanisms. Additionally, we introduce two key technical innovations for the BTL framework: (1) Blink Data Generation - an automated annotation pipeline specifically optimized for blink data, and (2) BTL Reward – the first rule-based reward mechanism that enables reinforcement learning driven by both process and outcome. Building upon this framework, we develop a GUI agent model named BTL-UI, which demonstrates competitive performance across both static GUI understanding and dynamic interaction tasks in comprehensive benchmarks. These results provide conclusive empirical validation of the framework’s efficacy in developing advanced GUI Agents.

Method: Introduces RLGF with Latent-Space Windowing Optimization for targeted feedback during diffusion and Hierarchical Geometric Reward system for multi-level geometric alignment (point-line-plane) and scene occupancy coherence.

Result: Applied to DiVE on nuScenes, RLGF reduces geometric errors (VP error by 21%, Depth error by 57%) and improves 3D object detection mAP by 12.7%, narrowing the gap to real-data performance.

Conclusion: RLGF provides a plug-and-play solution for generating geometrically sound synthetic videos for autonomous driving development.

Abstract: Synthetic data is crucial for advancing autonomous driving (AD) systems, yet current state-of-the-art video generation models, despite their visual realism, suffer from subtle geometric distortions that limit their utility for downstream perception tasks. We identify and quantify this critical issue, demonstrating a significant performance gap in 3D object detection when using synthetic versus real data. To address this, we introduce Reinforcement Learning with Geometric Feedback (RLGF), RLGF uniquely refines video diffusion models by incorporating rewards from specialized latent-space AD perception models. Its core components include an efficient Latent-Space Windowing Optimization technique for targeted feedback during diffusion, and a Hierarchical Geometric Reward (HGR) system providing multi-level rewards for point-line-plane alignment, and scene occupancy coherence. To quantify these distortions, we propose GeoScores. Applied to models like DiVE on nuScenes, RLGF substantially reduces geometric errors (e.g., VP error by 21%, Depth error by 57%) and dramatically improves 3D object detection mAP by 12.7%, narrowing the gap to real-data performance. RLGF offers a plug-and-play solution for generating geometrically sound and reliable synthetic videos for AD development.

Method: Proposes Condition-Aware Reparameterization for Flow Matching (CAR-Flow) - a learned shift that conditions the source, target, or both distributions to shorten the probability path the model must learn.

Result: On ImageNet-256, CAR-Flow reduces FID from 2.07 to 1.68 when equipped with SiT-XL/2, while adding less than 0.6% additional parameters. Visual and quantitative improvements shown on synthetic data.

Conclusion: CAR-Flow effectively improves flow-based conditional generative modeling by shortening learning paths through lightweight conditional reparameterization, achieving better performance with minimal computational overhead.

Abstract: Conditional generative modeling aims to learn a conditional data distribution from samples containing data-condition pairs. For this, diffusion and flow-based methods have attained compelling results. These methods use a learned (flow) model to transport an initial standard Gaussian noise that ignores the condition to the conditional data distribution. The model is hence required to learn both mass transport and conditional injection. To ease the demand on the model, we propose Condition-Aware Reparameterization for Flow Matching (CAR-Flow) – a lightweight, learned shift that conditions the source, the target, or both distributions. By relocating these distributions, CAR-Flow shortens the probability path the model must learn, leading to faster training in practice. On low-dimensional synthetic data, we visualize and quantify the effects of CAR-Flow. On higher-dimensional natural image data (ImageNet-256), equipping SiT-XL/2 with CAR-Flow reduces FID from 2.07 to 1.68, while introducing less than 0.6% additional parameters.

Method: The survey categorizes significant publications in supervised person Re-ID and explores latest advancements in unsupervised methods over the past three years.

Result: Supervised approaches show little room for further improvement, while unsupervised techniques have shown promising developments and narrowing performance gap with supervised methods.

Conclusion: The survey contributes to understanding both the mature supervised landscape and the emerging potential of unsupervised learning in person re-identification, suggesting potential convergence between the two paradigms.

Abstract: Person re-identification (ReId), a crucial task in surveillance, involves matching individuals across different camera views. The advent of Deep Learning, especially supervised techniques like Convolutional Neural Networks and Attention Mechanisms, has significantly enhanced person Re-ID. However, the success of supervised approaches hinges on vast amounts of annotated data, posing scalability challenges in data labeling and computational costs. To address these limitations, recent research has shifted towards unsupervised person re-identification. Leveraging abundant unlabeled data, unsupervised methods aim to overcome the need for pairwise labelled data. Although traditionally trailing behind supervised approaches, unsupervised techniques have shown promising developments in recent years, signalling a narrowing performance gap. Motivated by this evolving landscape, our survey pursues two primary objectives. First, we review and categorize significant publications in supervised person re-identification, providing an in-depth overview of the current state-of-the-art and emphasizing little room for further improvement in this domain. Second, we explore the latest advancements in unsupervised person re-identification over the past three years, offering insights into emerging trends and shedding light on the potential convergence of performance between supervised and unsupervised paradigms. This dual-focus survey aims to contribute to the evolving narrative of person re-identification, capturing both the mature landscape of supervised techniques and the promising outcomes in the realm of unsupervised learning.

Method: 1) Establish S²NN baseline using kernel clustering patterns from trained binary SNNs; 2) OS-Quant method to mitigate outlier-induced codeword selection bias via outlier identification and adaptive scaling; 3) MPFD method using membrane potential-based feature distillation for improved performance.

Result: S²NNs outperform existing quantized SNNs in both performance and efficiency on vision tasks, demonstrating superior compression and acceleration capabilities.

Conclusion: S²NNs with sub-bit weight representation are promising for edge computing applications, achieving better performance and efficiency than current quantized SNN approaches.

Abstract: Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence, but their continued scaling poses challenges for resource-limited deployment. Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks. To further explore the compression and acceleration potential of SNNs, we propose Sub-bit Spiking Neural Networks (S$^2$NNs) that represent weights with less than one bit. Specifically, we first establish an S$^2$NN baseline by leveraging the clustering patterns of kernels in well-trained binary SNNs. This baseline is highly efficient but suffers from \textit{outlier-induced codeword selection bias} during training. To mitigate this issue, we propose an \textit{outlier-aware sub-bit weight quantization} (OS-Quant) method, which optimizes codeword selection by identifying and adaptively scaling outliers. Furthermore, we propose a \textit{membrane potential-based feature distillation} (MPFD) method, improving the performance of highly compressed S$^2$NN via more precise guidance from a teacher model. Extensive results on vision tasks reveal that S$^2$NN outperforms existing quantized SNNs in both performance and efficiency, making it promising for edge computing applications.

Method: Extracts patch features using self-supervised DINO ViT encoder, builds affinity matrix, applies spectral clustering with automatic segment count selection via eigengap silhouette search, and sharpens boundaries with DenseCRF.

Result: Achieves competitive mIoU and pixel accuracy on COCO Stuff and ADE20K datasets, matching recent unsupervised baselines despite zero training.

Conclusion: CLASP provides a strong, easily reproducible baseline for unsupervised image segmentation that works well on large unannotated corpora without requiring any training or labeled data.

Abstract: We introduce CLASP (Clustering via Adaptive Spectral Processing), a lightweight framework for unsupervised image segmentation that operates without any labeled data or finetuning. CLASP first extracts per patch features using a self supervised ViT encoder (DINO); then, it builds an affinity matrix and applies spectral clustering. To avoid manual tuning, we select the segment count automatically with a eigengap silhouette search, and we sharpen the boundaries with a fully connected DenseCRF. Despite its simplicity and training free nature, CLASP attains competitive mIoU and pixel accuracy on COCO Stuff and ADE20K, matching recent unsupervised baselines. The zero training design makes CLASP a strong, easily reproducible baseline for large unannotated corpora especially common in digital advertising and marketing workflows such as brand safety screening, creative asset curation, and social media content moderation

Method: Proposes Non-Linear Projections of the Null-Space (NPN) - a regularization that uses neural networks to promote solutions in low-dimensional projections of the sensing matrix’s null-space rather than enforcing image-domain constraints.

Result: NPN priors consistently enhance reconstruction fidelity across various imaging inverse problems (compressive sensing, deblurring, super-resolution, CT, MRI) with different reconstruction frameworks including plug-and-play methods, unrolling networks, deep image prior, and diffusion models.

Conclusion: NPN provides interpretable and flexible regularization by focusing on null-space structure, is adaptable to various inverse problems, compatible with existing frameworks, and complementary to conventional image-domain priors with theoretical convergence guarantees.

Abstract: Imaging inverse problems aim to recover high-dimensional signals from undersampled, noisy measurements, a fundamentally ill-posed task with infinite solutions in the null-space of the sensing operator. To resolve this ambiguity, prior information is typically incorporated through handcrafted regularizers or learned models that constrain the solution space. However, these priors typically ignore the task-specific structure of that null-space. In this work, we propose Non-Linear Projections of the Null-Space (NPN), a novel class of regularization that, instead of enforcing structural constraints in the image domain, promotes solutions that lie in a low-dimensional projection of the sensing matrix’s null-space with a neural network. Our approach has two key advantages: (1) Interpretability: by focusing on the structure of the null-space, we design sensing-matrix-specific priors that capture information orthogonal to the signal components that are fundamentally blind to the sensing process. (2) Flexibility: NPN is adaptable to various inverse problems, compatible with existing reconstruction frameworks, and complementary to conventional image-domain priors. We provide theoretical guarantees on convergence and reconstruction accuracy when used within plug-and-play methods. Empirical results across diverse sensing matrices demonstrate that NPN priors consistently enhance reconstruction fidelity in various imaging inverse problems, such as compressive sensing, deblurring, super-resolution, computed tomography, and magnetic resonance imaging, with plug-and-play methods, unrolling networks, deep image prior, and diffusion models.

Method: Uses inpainting-guided perturbation with SAM-refined masks for object removal/replacement and background replacement, tested on YOLOv9 for harbor seal detection in drone imagery.

Result: Produces photorealistic explanations that localize diagnostic structures, avoid deletion artifacts, and provide domain-relevant insights validated by expert review and re-scoring metrics.

Conclusion: The approach supports trustworthy AI deployment in ecology by generating interpretable, ecologically plausible explanations that reveal fine-grained morphological cues driving model predictions.

Abstract: Ecological monitoring is increasingly automated by vision models, yet opaque predictions limit trust and field adoption. We present an inpainting-guided, perturbation-based explanation technique that produces photorealistic, mask-localized edits that preserve scene context. Unlike masking or blurring, these edits stay in-distribution and reveal which fine-grained morphological cues drive predictions in tasks such as species recognition and trait attribution. We demonstrate the approach on a YOLOv9 detector fine-tuned for harbor seal detection in Glacier Bay drone imagery, using Segment-Anything-Model-refined masks to support two interventions: (i) object removal/replacement (e.g., replacing seals with plausible ice/water or boats) and (ii) background replacement with original animals composited onto new scenes. Explanations are assessed by re-scoring perturbed images (flip rate, confidence drop) and by expert review for ecological plausibility and interpretability. The resulting explanations localize diagnostic structures, avoid deletion artifacts common to traditional perturbations, and yield domain-relevant insights that support expert validation and more trustworthy deployment of AI in ecology.

Method: An architecture that leverages the spatial understanding and inductive bias of 3D foundation models to match segments across image pairs with extreme viewpoint changes.

Result: Outperforms state-of-the-art methods including SAM2 video propagator and local feature matching by up to 30% on AUPRC metric on ScanNet++ and Replica datasets.

Conclusion: The proposed approach effectively handles wide-baseline segment matching with extreme viewpoint changes and shows benefits for downstream tasks like 3D instance mapping and object-relative navigation.

Abstract: Segment matching is an important intermediate task in computer vision that establishes correspondences between semantically or geometrically coherent regions across images. Unlike keypoint matching, which focuses on localized features, segment matching captures structured regions, offering greater robustness to occlusions, lighting variations, and viewpoint changes. In this paper, we leverage the spatial understanding of 3D foundation models to tackle wide-baseline segment matching, a challenging setting involving extreme viewpoint shifts. We propose an architecture that uses the inductive bias of these 3D foundation models to match segments across image pairs with up to 180 degree view-point change rotation. Extensive experiments show that our approach outperforms state-of-the-art methods, including the SAM2 video propagator and local feature matching methods, by up to 30% on the AUPRC metric, on ScanNet++ and Replica datasets. We further demonstrate benefits of the proposed model on relevant downstream tasks, including 3D instance mapping and object-relative navigation. Project Page: https://segmast3r.github.io/

Method: Two-stage framework: Stage One introduces LH-VAE with visual semantic constraints, progressive degradation perturbations, and latent equivariance. Stage Two jointly trains the refined VAE with restoration model using High-Frequency Low-Rank Adaptation (HF-LoRA) - encoder LoRA with fidelity-oriented loss and decoder LoRA with perception-oriented loss, trained via alternating optimization.

Result: Achieves state-of-the-art performance across UHD and standard-resolution tasks, effectively balancing efficiency, perceptual quality, and reconstruction accuracy with tunable fidelity-perception trade-offs.

Conclusion: Latent Harmony successfully redefines VAEs for UHD restoration by jointly regularizing latent space and enforcing high-frequency-aware reconstruction, overcoming the limitations of traditional VAE approaches.

Abstract: Ultra-High Definition (UHD) image restoration faces a trade-off between computational efficiency and high-frequency detail retention. While Variational Autoencoders (VAEs) improve efficiency via latent-space processing, their Gaussian constraint often discards degradation-specific high-frequency information, hurting reconstruction fidelity. To overcome this, we propose Latent Harmony, a two-stage framework that redefines VAEs for UHD restoration by jointly regularizing the latent space and enforcing high-frequency-aware reconstruction.In Stage One, we introduce LH-VAE, which enhances semantic robustness through visual semantic constraints and progressive degradation perturbations, while latent equivariance strengthens high-frequency reconstruction.Stage Two jointly trains this refined VAE with a restoration model using High-Frequency Low-Rank Adaptation (HF-LoRA): an encoder LoRA guided by a fidelity-oriented high-frequency alignment loss to recover authentic details, and a decoder LoRA driven by a perception-oriented loss to synthesize realistic textures. Both LoRA modules are trained via alternating optimization with selective gradient propagation to preserve the pretrained latent structure.At inference, a tunable parameter {\alpha} enables flexible fidelity-perception trade-offs.Experiments show Latent Harmony achieves state-of-the-art performance across UHD and standard-resolution tasks, effectively balancing efficiency, perceptual quality, and reconstruction accuracy.

Method: Models camera ego-motion as a continuous spline and optical flow as an implicit neural representation, embedding spatial-temporal coherence through inductive biases. Incorporates structure-and-motion priors via differential geometric constraints without explicit depth estimation, maintaining geometric consistency.

Result: Achieves state-of-the-art performance among unsupervised methods and competitive results even with supervised approaches. Demonstrates versatility in general 6-DoF motion scenarios.

Conclusion: The proposed E-MoFlow framework successfully unifies ego-motion and optical flow estimation through implicit regularization under a fully unsupervised paradigm, overcoming limitations of existing methods while maintaining geometric rigor.

Abstract: The estimation of optical flow and 6-DoF ego-motion, two fundamental tasks in 3D vision, has typically been addressed independently. For neuromorphic vision (e.g., event cameras), however, the lack of robust data association makes solving the two problems separately an ill-posed challenge, especially in the absence of supervision via ground truth. Existing works mitigate this ill-posedness by either enforcing the smoothness of the flow field via an explicit variational regularizer or leveraging explicit structure-and-motion priors in the parametrization to improve event alignment. The former notably introduces bias in results and computational overhead, while the latter, which parametrizes the optical flow in terms of the scene depth and the camera motion, often converges to suboptimal local minima. To address these issues, we propose an unsupervised framework that jointly optimizes egomotion and optical flow via implicit spatial-temporal and geometric regularization. First, by modeling camera’s egomotion as a continuous spline and optical flow as an implicit neural representation, our method inherently embeds spatial-temporal coherence through inductive biases. Second, we incorporate structure-and-motion priors through differential geometric constraints, bypassing explicit depth estimation while maintaining rigorous geometric consistency. As a result, our framework (called E-MoFlow) unifies egomotion and optical flow estimation via implicit regularization under a fully unsupervised paradigm. Experiments demonstrate its versatility to general 6-DoF motion scenarios, achieving state-of-the-art performance among unsupervised methods and competitive even with supervised approaches.

Method: A reconstruction-based framework guided by ’less is more’ philosophy, orchestrating five simple elements to achieve superior performance without modification across diverse tasks.

Result: Achieves unprecedented 99.9% and 99.3% I-AUROC on MVTec-AD and VisA for multi-class models, state-of-the-art performance in multi-view/multi-modal inspection, and surpasses previous full-shot models using only 8 normal examples per class.

Conclusion: Dinomaly2’s minimalistic design, computational scalability, and universal applicability position it as a unified solution for the full spectrum of real-world anomaly detection applications.

Abstract: Unsupervised anomaly detection (UAD) has evolved from building specialized single-class models to unified multi-class models, yet existing multi-class models significantly underperform the most advanced one-for-one counterparts. Moreover, the field has fragmented into specialized methods tailored to specific scenarios (multi-class, 3D, few-shot, etc.), creating deployment barriers and highlighting the need for a unified solution. In this paper, we present Dinomaly2, the first unified framework for full-spectrum image UAD, which bridges the performance gap in multi-class models while seamlessly extending across diverse data modalities and task settings. Guided by the “less is more” philosophy, we demonstrate that the orchestration of five simple element achieves superior performance in a standard reconstruction-based framework. This methodological minimalism enables natural extension across diverse tasks without modification, establishing that simplicity is the foundation of true universality. Extensive experiments on 12 UAD benchmarks demonstrate Dinomaly2’s full-spectrum superiority across multiple modalities (2D, multi-view, RGB-3D, RGB-IR), task settings (single-class, multi-class, inference-unified multi-class, few-shot) and application domains (industrial, biological, outdoor). For example, our multi-class model achieves unprecedented 99.9% and 99.3% image-level (I-) AUROC on MVTec-AD and VisA respectively. For multi-view and multi-modal inspection, Dinomaly2 demonstrates state-of-the-art performance with minimum adaptations. Moreover, using only 8 normal examples per class, our method surpasses previous full-shot models, achieving 98.7% and 97.4% I-AUROC on MVTec-AD and VisA. The combination of minimalistic design, computational scalability, and universal applicability positions Dinomaly2 as a unified solution for the full spectrum of real-world anomaly detection applications.

Method: Uses panoramic video generation (RGB, semantics, depth, 3D occupancy), normalized Plucker ray-map for trajectory encoding, and 3D occupancy-based rule rewards for driving compliance.

Result: Achieves state-of-the-art performance in video generation, control accuracy, and long-horizon stability, providing reliable closed-loop evaluation.

Conclusion: OmniNWM successfully unifies state, action, and reward modeling in autonomous driving through its panoramic approach and occupancy-grounded rewards.

Abstract: Autonomous driving world models are expected to work effectively across three core dimensions: state, action, and reward. Existing models, however, are typically restricted to limited state modalities, short video sequences, imprecise action control, and a lack of reward awareness. In this paper, we introduce OmniNWM, an omniscient panoramic navigation world model that addresses all three dimensions within a unified framework. For state, OmniNWM jointly generates panoramic videos of RGB, semantics, metric depth, and 3D occupancy. A flexible forcing strategy enables high-quality long-horizon auto-regressive generation. For action, we introduce a normalized panoramic Plucker ray-map representation that encodes input trajectories into pixel-level signals, enabling highly precise and generalizable control over panoramic video generation. Regarding reward, we move beyond learning reward functions with external image-based models: instead, we leverage the generated 3D occupancy to directly define rule-based dense rewards for driving compliance and safety. Extensive experiments demonstrate that OmniNWM achieves state-of-the-art performance in video generation, control accuracy, and long-horizon stability, while providing a reliable closed-loop evaluation framework through occupancy-grounded rewards. Project page is available at https://arlo0o.github.io/OmniNWM/.

Method: Uses geometric arguments and fundamental pattern matching principles for derivation, and proposes steerable convolution layers with interpolation kernels.

Result: Developed a more intuitive understanding of steerable CNNs, including explanations for Clebsch-Gordan decomposition and spherical harmonics, and created improved convolution layers.

Conclusion: The proposed geometric approach offers intuitive insights into steerable CNNs and the interpolation kernel method provides enhanced implementation with better noise robustness compared to existing methods.

Abstract: In contrast to the somewhat abstract, group theoretical approach adopted by many papers, our work provides a new and more intuitive derivation of steerable convolutional neural networks in $d$ dimensions. This derivation is based on geometric arguments and fundamental principles of pattern matching. We offer an intuitive explanation for the appearance of the Clebsch–Gordan decomposition and spherical harmonic basis functions. Furthermore, we suggest a novel way to construct steerable convolution layers using interpolation kernels that improve upon existing implementation, and offer greater robustness to noisy data.

Method: Dream4Drive decomposes input videos into 3D-aware guidance maps, renders 3D assets onto these maps, and fine-tunes a driving world model to produce edited multi-view photorealistic videos for training perception models. It also includes DriveObj3D, a large-scale 3D asset dataset.

Result: The framework enables unprecedented flexibility in generating multi-view corner cases at scale, significantly boosting corner case perception in autonomous driving. Experiments show Dream4Drive effectively boosts downstream perception model performance under various training epochs.

Conclusion: Dream4Drive provides a novel approach to synthetic data generation that genuinely enhances autonomous driving perception tasks, addressing the limitations of existing methods and demonstrating clear benefits for downstream model performance.

Abstract: Recent advancements in driving world models enable controllable generation of high-quality RGB videos or multimodal videos. Existing methods primarily focus on metrics related to generation quality and controllability. However, they often overlook the evaluation of downstream perception tasks, which are $\mathbf{really\ crucial}$ for the performance of autonomous driving. Existing methods usually leverage a training strategy that first pretrains on synthetic data and finetunes on real data, resulting in twice the epochs compared to the baseline (real data only). When we double the epochs in the baseline, the benefit of synthetic data becomes negligible. To thoroughly demonstrate the benefit of synthetic data, we introduce Dream4Drive, a novel synthetic data generation framework designed for enhancing the downstream perception tasks. Dream4Drive first decomposes the input video into several 3D-aware guidance maps and subsequently renders the 3D assets onto these guidance maps. Finally, the driving world model is fine-tuned to produce the edited, multi-view photorealistic videos, which can be used to train the downstream perception models. Dream4Drive enables unprecedented flexibility in generating multi-view corner cases at scale, significantly boosting corner case perception in autonomous driving. To facilitate future research, we also contribute a large-scale 3D asset dataset named DriveObj3D, covering the typical categories in driving scenarios and enabling diverse 3D-aware video editing. We conduct comprehensive experiments to show that Dream4Drive can effectively boost the performance of downstream perception models under various training epochs. Page: https://wm-research.github.io/Dream4Drive/ GitHub Link: https://github.com/wm-research/Dream4Drive

Method: Uses Generative Communication with Deformable Message Extractor to extract spatial messages, Spatial-Aware Feature Generator with conditional diffusion model to generate aligned features, and Channel Enhancer for refinement before fusion.

Result: Outperforms state-of-the-art methods on OPV2V-H, DAIR-V2X and V2X-Real datasets with 81% reduction in computational cost and parameter count when incorporating new agents.

Conclusion: GenComm provides an effective solution for heterogeneous multi-agent collaboration through non-intrusive feature generation and lightweight spatial alignment, enabling scalable and efficient perception across diverse agents.

Abstract: Multi-agent collaboration enhances the perception capabilities of individual agents through information sharing. However, in real-world applications, differences in sensors and models across heterogeneous agents inevitably lead to domain gaps during collaboration. Existing approaches based on adaptation and reconstruction fail to support pragmatic heterogeneous collaboration due to two key limitations: (1) Intrusive retraining of the encoder or core modules disrupts the established semantic consistency among agents; and (2) accommodating new agents incurs high computational costs, limiting scalability. To address these challenges, we present a novel Generative Communication mechanism (GenComm) that facilitates seamless perception across heterogeneous multi-agent systems through feature generation, without altering the original network, and employs lightweight numerical alignment of spatial information to efficiently integrate new agents at minimal cost. Specifically, a tailored Deformable Message Extractor is designed to extract spatial message for each collaborator, which is then transmitted in place of intermediate features. The Spatial-Aware Feature Generator, utilizing a conditional diffusion model, generates features aligned with the ego agent’s semantic space while preserving the spatial information of the collaborators. These generated features are further refined by a Channel Enhancer before fusion. Experiments conducted on the OPV2V-H, DAIR-V2X and V2X-Real datasets demonstrate that GenComm outperforms existing state-of-the-art methods, achieving an 81% reduction in both computational cost and parameter count when incorporating new agents. Our code is available at https://github.com/jeffreychou777/GenComm.

Method: Proposes Suspicion Progression Analysis Network (SPAN) with suspicion score formula based on Temporal Point Process theory, Suspicion Coefficient Modulation using multimodal information, and Concept-Anchored Mapping to link actions to intention concepts.

Result: SPAN significantly outperforms existing methods on HAI dataset, reducing MSE by 19.8% and improving average mAP by 1.78%, with 2.74% mAP gain in low-frequency cases.

Conclusion: Continuous suspicion modeling enables earlier detection and proactive intervention, enhancing system explainability and practical utility in security applications compared to discrete classification.

Abstract: Temporal Intention Localization (TIL) is crucial for video surveillance, focusing on identifying varying levels of suspicious intentions to improve security monitoring. However, existing discrete classification methods fail to capture the continuous nature of suspicious intentions, limiting early intervention and explainability. In this paper, we propose the Suspicion Progression Analysis Network (SPAN), which shifts from discrete classification to continuous regression, enabling the capture of fluctuating and evolving suspicious intentions. We reveal that suspicion exhibits long-term dependencies and cumulative effects, similar to Temporal Point Process (TPP) theory. Based on these insights, we define a suspicion score formula that models continuous changes while accounting for temporal characteristics. We also introduce Suspicion Coefficient Modulation, which adjusts suspicion coefficients using multimodal information to reflect the varying impacts of suspicious actions. Additionally, the Concept-Anchored Mapping method is proposed to link suspicious actions to predefined intention concepts, offering insights into both the actions and their potential underlying intentions. Extensive experiments on the HAI dataset show that SPAN significantly outperforms existing methods, reducing MSE by 19.8% and improving average mAP by 1.78%. Notably, SPAN achieves a 2.74% mAP gain in low-frequency cases, demonstrating its superior ability to capture subtle behavioral changes. Compared to discrete classification systems, our continuous suspicion modeling approach enables earlier detection and proactive intervention, greatly enhancing system explainability and practical utility in security applications.

Method: Proposes EditInfinity using Infinity (binary-quantized generative model) with efficient image inversion mechanism integrating text prompting rectification and style preservation, plus holistic smoothing strategy.

Result: Extensive experiments on PIE-Bench benchmark across add/change/delete operations demonstrate superior performance compared to state-of-the-art diffusion-based baselines.

Conclusion: VQ-based generative models with exact intermediate representations enable more effective supervision for precise image inversion, leading to better text-driven image editing with high fidelity and semantic alignment.

Abstract: Adapting pretrained diffusion-based generative models for text-driven image editing with negligible tuning overhead has demonstrated remarkable potential. A classical adaptation paradigm, as followed by these methods, first infers the generative trajectory inversely for a given source image by image inversion, then performs image editing along the inferred trajectory guided by the target text prompts. However, the performance of image editing is heavily limited by the approximation errors introduced during image inversion by diffusion models, which arise from the absence of exact supervision in the intermediate generative steps. To circumvent this issue, we investigate the parameter-efficient adaptation of VQ-based generative models for image editing, and leverage their inherent characteristic that the exact intermediate quantized representations of a source image are attainable, enabling more effective supervision for precise image inversion. Specifically, we propose \emph{EditInfinity}, which adapts \emph{Infinity}, a binary-quantized generative model, for image editing. We propose an efficient yet effective image inversion mechanism that integrates text prompting rectification and image style preservation, enabling precise image inversion. Furthermore, we devise a holistic smoothing strategy which allows our \emph{EditInfinity} to perform image editing with high fidelity to source images and precise semantic alignment to the text prompts. Extensive experiments on the PIE-Bench benchmark across “add”, “change”, and “delete” editing operations, demonstrate the superior performance of our model compared to state-of-the-art diffusion-based baselines. Code available at: https://github.com/yx-chen-ust/EditInfinity.

Method: Uses MLLMs in multi-agent framework with perceptual extraction, human feedback, schema validation, and automated BIM scripting to convert sketches to structured JSON layouts, then to executable BIM scripts.

Result: Experiments on 10 floor plans show strong convergence: openings captured with high reliability initially, wall detection starts at 83% and reaches near-perfect alignment after feedback. All metrics (precision, recall, F1) above 0.83, geometric errors decrease to zero through feedback.

Conclusion: MLLM-driven multi-agent reasoning can effectively automate BIM creation from freehand sketches, making the process accessible to users without technical BIM expertise.

Abstract: This study introduces a human-in-the-loop pipeline that converts unscaled, hand-drawn floor plan sketches into semantically consistent 3D BIM models. The workflow leverages multimodal large language models (MLLMs) within a multi-agent framework, combining perceptual extraction, human feedback, schema validation, and automated BIM scripting. Initially, sketches are iteratively refined into a structured JSON layout of walls, doors, and windows. Later, these layouts are transformed into executable scripts that generate 3D BIM models. Experiments on ten diverse floor plans demonstrate strong convergence: openings (doors, windows) are captured with high reliability in the initial pass, while wall detection begins around 83% and achieves near-perfect alignment after a few feedback iterations. Across all categories, precision, recall, and F1 scores remain above 0.83, and geometric errors (RMSE, MAE) progressively decrease to zero through feedback corrections. This study demonstrates how MLLM-driven multi-agent reasoning can make BIM creation accessible to both experts and non-experts using only freehand sketches.

Method: CAS uses two GPT-2 models fine-tuned on WikiArt concepts: a Concept Coherence Model that scores concept co-occurrence plausibility, and a Cultural Context Model that estimates typicality within artists’ work. It selects high-coherence, low-typicality combinations.

Result: Human evaluation (N=100) showed CAS outperforms random selection and GPT-4o baselines, achieving performance comparable to human art students in both originality and harmony. Quantitative analysis revealed more diverse outputs and broader conceptual exploration than GPT-4o.

Conclusion: Artificial cultural alienness can unlock creative potential in autonomous agents by maintaining internal consistency while deviating from learned conventions.

Abstract: In open-ended domains like art, autonomous agents must generate ideas that are both original and internally coherent, yet current Large Language Models (LLMs) either default to familiar cultural patterns or sacrifice coherence when pushed toward novelty. We address this by introducing the Cultural Alien Sampler (CAS), a concept-selection method that explicitly separates compositional fit from cultural typicality. CAS uses two GPT-2 models fine-tuned on WikiArt concepts: a Concept Coherence Model that scores whether concepts plausibly co-occur within artworks, and a Cultural Context Model that estimates how typical those combinations are within individual artists’ bodies of work. CAS targets combinations that are high in coherence and low in typicality, yielding ideas that maintain internal consistency while deviating from learned conventions and embedded cultural context. In a human evaluation (N = 100), our approach outperforms random selection and GPT-4o baselines and achieves performance comparable to human art students in both perceived originality and harmony. Additionally, a quantitative study shows that our method produces more diverse outputs and explores a broader conceptual space than its GPT-4o counterpart, demonstrating that artificial cultural alienness can unlock creative potential in autonomous agents.

Method: The paper introduces extensional fuzzy numbers and defines operations and relational operators (=, >, >=, <, <=) for them. It provides a C++ implementation available on GitHub.

Result: The proposed approach is demonstrated through several applicational examples showing improved computational efficiency and shape preservation compared to traditional fuzzy number operations.

Conclusion: Extensional fuzzy numbers offer a viable alternative to traditional fuzzy sets that addresses key limitations in computational complexity and shape preservation, potentially expanding the application field of fuzzy numbers.

Abstract: Fuzzy numbers are commonly represented with fuzzy sets. Their objective is to better represent imprecise data. However, operations on fuzzy numbers are not as straightforward as maths on crisp numbers. Commonly, the Zadeh’s extension rule is applied to elaborate a result. This can produce two problems: (1) high computational complexity and (2) for some fuzzy sets and some operations the results is not a fuzzy set with the same features (eg. multiplication of two triangular fuzzy sets does not produce a triangular fuzzy set). One more problem is the fuzzy spread – fuzziness of the result increases with the number of operations. These facts can severely limit the application field of fuzzy numbers. In this paper we would like to revisite this problem with a different kind of fuzzy numbers – extensional fuzzy numbers. The paper defines operations on extensional fuzzy numbers and relational operators (=, >, >=, <, <=) for them. The proposed approach is illustrated with several applicational examples. The C++ implementation is available from a public GitHub repository.

Method: Customize open-source LLMs (Llama, Qwen, Gemma, MedGemma) to extract unified MOUD prescription attributes, process records in fixed JSON schema, perform normalization and consistency checks, and compute standardized MOUD days metric.

Result: Larger models performed best: Qwen2.5-32B achieved 93.4% coverage with 93.0% exact-match accuracy, and MedGemma-27B attained 93.1%/92.2% across five clinics (25,605 records from 1,257 patients).

Conclusion: This approach enables consistent cross-site analyses of MOUD exposure, adherence, and retention by removing site-specific ETL and supporting privacy-preserving deployment.

Abstract: Harmonizing medication data across Electronic Health Record (EHR) systems is a persistent barrier to monitoring medications for opioid use disorder (MOUD). In heterogeneous EHR systems, key prescription attributes are scattered across differently formatted fields and freetext notes. We present a practical framework that customizes open source large language models (LLMs), including Llama, Qwen, Gemma, and MedGemma, to extract a unified set of MOUD prescription attributes (prescription date, drug name, duration, total quantity, daily quantity, and refills) from heterogeneous, site specific data and compute a standardized metric of medication coverage, \emph{MOUD days}, per patient. Our pipeline processes records directly in a fixed JSON schema, followed by lightweight normalization and cross-field consistency checks. We evaluate the system on prescription level EHR data from five clinics in a national OUD study (25{,}605 records from 1{,}257 patients), using a previously annotated benchmark of 10{,}369 records (776 patients) as the ground truth. Performance is reported as coverage (share of records with a valid, matchable output) and record-level exact-match accuracy. Larger models perform best overall: Qwen2.5-32B achieves \textbf{93.4%} coverage with \textbf{93.0%} exact-match accuracy across clinics, and MedGemma-27B attains \textbf{93.1%}/\textbf{92.2%}. A brief error review highlights three common issues and fixes: imputing missing dosage fields using within-drug norms, handling monthly/weekly injectables (e.g., Vivitrol) by setting duration from the documented schedule, and adding unit checks to prevent mass units (e.g., ``250 g’’) from being misread as daily counts. By removing brittle, site-specific ETL and supporting local, privacy-preserving deployment, this approach enables consistent cross-site analyses of MOUD exposure, adherence, and retention in real-world settings.

Method: Develops AI Preemptionism as a view where AEA outputs replace human judgment, then critiques it using classic objections (uncritical deference, epistemic entrenchment, unhinging epistemic bases). Proposes an alternative total evidence view where AI outputs serve as contributory reasons alongside human considerations.

Result: The total evidence view offers three advantages: mitigates expertise atrophy by keeping humans engaged, provides epistemic justification for human oversight and control, and explains justified mistrust when reliability conditions are unmet.

Conclusion: While demanding in practice, the total evidence view provides a principled framework for determining justified AI deference, especially in high-stakes contexts requiring rigorous reliability assessment.

Abstract: When should we defer to AI outputs over human expert judgment? Drawing on recent work in social epistemology, I motivate the idea that some AI systems qualify as Artificial Epistemic Authorities (AEAs) due to their demonstrated reliability and epistemic superiority. I then introduce AI Preemptionism, the view that AEA outputs should replace rather than supplement a user’s independent epistemic reasons. I show that classic objections to preemptionism

• such as uncritical deference, epistemic entrenchment, and unhinging epistemic bases - apply in amplified form to AEAs, given their opacity, self-reinforcing authority, and lack of epistemic failure markers. Against this, I develop a more promising alternative: a total evidence view of AI deference. According to this view, AEA outputs should function as contributory reasons rather than outright replacements for a user’s independent epistemic considerations. This approach has three key advantages: (i) it mitigates expertise atrophy by keeping human users engaged, (ii) it provides an epistemic case for meaningful human oversight and control, and (iii) it explains the justified mistrust of AI when reliability conditions are unmet. While demanding in practice, this account offers a principled way to determine when AI deference is justified, particularly in high-stakes contexts requiring rigorous reliability.

Method: A three-stage approach: (1) Tool Invocation with Evidence-driven Validator for output normalization and verification, (2) Diagnostic Planning that assembles expert teams based on task requirements, (3) Collaborative Decision-making that integrates expert insights with contextual memories.

Result: Experiments show CXRAgent delivers strong performance on various CXR interpretation tasks, provides visual evidence, and generalizes well to clinical tasks of different complexity.

Conclusion: CXRAgent represents an effective multi-stage agent framework that enhances CXR interpretation through reliable tool coordination, adaptive diagnostic planning, and collaborative reasoning, improving both performance and credibility.

Abstract: Chest X-ray (CXR) plays a pivotal role in clinical diagnosis, and a variety of task-specific and foundation models have been developed for automatic CXR interpretation. However, these models often struggle to adapt to new diagnostic tasks and complex reasoning scenarios. Recently, LLM-based agent models have emerged as a promising paradigm for CXR analysis, enhancing model’s capability through tool coordination, multi-step reasoning, and team collaboration, etc. However, existing agents often rely on a single diagnostic pipeline and lack mechanisms for assessing tools’ reliability, limiting their adaptability and credibility. To this end, we propose CXRAgent, a director-orchestrated, multi-stage agent for CXR interpretation, where a central director coordinates the following stages: (1) Tool Invocation: The agent strategically orchestrates a set of CXR-analysis tools, with outputs normalized and verified by the Evidence-driven Validator (EDV), which grounds diagnostic outputs with visual evidence to support reliable downstream diagnosis; (2) Diagnostic Planning: Guided by task requirements and intermediate findings, the agent formulates a targeted diagnostic plan. It then assembles an expert team accordingly, defining member roles and coordinating their interactions to enable adaptive and collaborative reasoning; (3) Collaborative Decision-making: The agent integrates insights from the expert team with accumulated contextual memories, synthesizing them into an evidence-backed diagnostic conclusion. Experiments on various CXR interpretation tasks show that CXRAgent delivers strong performance, providing visual evidence and generalizes well to clinical tasks of different complexity. Code and data are valuable at this \href{https://github.com/laojiahuo2003/CXRAgent/}{link}.

Method: Multi-agent framework with knowledge base, schema profiling, semantic enrichment, embeddings for context retrieval, and specialized agents for entity extraction, metadata retrieval, query logic formulation, SQL generation, and programmatic/semantic validation.

Result: 81.2% overall accuracy on KaggleDBQA (221/272 questions) and 87.7% accuracy on spatial queries (79/90 questions), compared to 76.7% without the review agent. System sometimes generates queries more semantically aligned with user intent than benchmark queries.

Conclusion: The framework makes spatial analysis more accessible and provides a robust foundation for spatial Text-to-SQL systems, advancing autonomous GIS development.

Abstract: The complexity of Structured Query Language (SQL) and the specialized nature of geospatial functions in tools like PostGIS present significant barriers to non-experts seeking to analyze spatial data. While Large Language Models (LLMs) offer promise for translating natural language into SQL (Text-to-SQL), single-agent approaches often struggle with the semantic and syntactic complexities of spatial queries. To address this, we propose a multi-agent framework designed to accurately translate natural language questions into spatial SQL queries. The framework integrates several innovative components, including a knowledge base with programmatic schema profiling and semantic enrichment, embeddings for context retrieval, and a collaborative multi-agent pipeline as its core. This pipeline comprises specialized agents for entity extraction, metadata retrieval, query logic formulation, SQL generation, and a review agent that performs programmatic and semantic validation of the generated SQL to ensure correctness (self-verification). We evaluate our system using both the non-spatial KaggleDBQA benchmark and a new, comprehensive SpatialQueryQA benchmark that includes diverse geometry types, predicates, and three levels of query complexity. On KaggleDBQA, the system achieved an overall accuracy of 81.2% (221 out of 272 questions) after the review agent’s review and corrections. For spatial queries, the system achieved an overall accuracy of 87.7% (79 out of 90 questions), compared with 76.7% without the review agent. Beyond accuracy, results also show that in some instances the system generates queries that are more semantically aligned with user intent than those in the benchmarks. This work makes spatial analysis more accessible, and provides a robust, generalizable foundation for spatial Text-to-SQL systems, advancing the development of autonomous GIS.

Method: Proposes multimodal Direct Preference Optimization (mDPO) for visual context alignment, Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture for routing queries to specialized experts, and federated governance with local meta-cognitive uncertainty estimator for adaptive Chain-of-Thought reasoning.

Result: Achieves state-of-the-art performance on three Med-VQA datasets, outperforming strong retrieval-augmented baselines by up to 11.85% in F1-score and reducing average reasoning length by 51.60% compared to fixed-depth CoT approaches.

Conclusion: MedAlign effectively addresses key challenges in medical LVLMs, providing visually accurate responses while enabling efficient reasoning and multi-institutional collaboration in clinical settings.

Abstract: Recently, large models have shown significant potential for smart healthcare. However, the deployment of Large Vision-Language Models (LVLMs) for clinical services is currently hindered by three critical challenges: a tendency to hallucinate answers not grounded in visual evidence, the inefficiency of fixed-depth reasoning, and the difficulty of multi-institutional collaboration. To address these challenges, in this paper, we develop MedAlign, a novel framework to ensure visually accurate LVLM responses for Medical Visual Question Answering (Med-VQA). Specifically, we first propose a multimodal Direct Preference Optimization (mDPO) objective to explicitly align preference learning with visual context. We then design a Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture that utilizes image and text similarity to route queries to a specialized and context-augmented LVLM (i.e., an expert), thereby mitigating hallucinations in LVLMs. To achieve adaptive reasoning and facilitate multi-institutional collaboration, we propose a federated governance mechanism, where the selected expert, fine-tuned on clinical datasets based on mDPO, locally performs iterative Chain-of-Thought (CoT) reasoning via the local meta-cognitive uncertainty estimator. Extensive experiments on three representative Med-VQA datasets demonstrate that MedAlign achieves state-of-the-art performance, outperforming strong retrieval-augmented baselines by up to $11.85%$ in F1-score, and simultaneously reducing the average reasoning length by $51.60%$ compared with fixed-depth CoT approaches.

Method: Builds on Deep Q-Network (DQN) to develop an algorithm that finds safe policies for worst-case environments compatible with observations, handling confounding biases.

Result: Applied to twelve confounded Atari games, consistently outperforms standard DQN in all games where observed input to behavioral and target policies mismatch and unobserved confounders exist.

Conclusion: The proposed method effectively handles confounding biases in reinforcement learning and demonstrates superior performance over standard DQN in confounded environments.

Abstract: A key task in Artificial Intelligence is learning effective policies for controlling agents in unknown environments to optimize performance measures. Off-policy learning methods, like Q-learning, allow learners to make optimal decisions based on past experiences. This paper studies off-policy learning from biased data in complex and high-dimensional domains where \emph{unobserved confounding} cannot be ruled out a priori. Building on the well-celebrated Deep Q-Network (DQN), we propose a novel deep reinforcement learning algorithm robust to confounding biases in observed data. Specifically, our algorithm attempts to find a safe policy for the worst-case environment compatible with the observations. We apply our method to twelve confounded Atari games, and find that it consistently dominates the standard DQN in all games where the observed input to the behavioral and target policies mismatch and unobserved confounders exist.

Method: Built an AI voter agent that interprets proposal contexts, retrieves historical deliberation data, and independently determines voting positions. Used 3K+ proposals from major protocols within a realistic financial simulation environment based on verifiable blockchain data, implemented through modular composable program (MCP) workflow via Agentics framework.

Result: The agent’s decisions showed strong alignment with human and token-weighted outcomes, measured by carefully designed evaluation metrics. The AI agent produced interpretable, auditable, and empirically grounded voting signals.

Conclusion: Agentic AI can effectively augment collective decision-making in DAO governance settings, contributing to the design of explainable and economically rigorous AI agents for decentralized financial systems.

Abstract: This paper presents a first empirical study of agentic AI as autonomous decision-makers in decentralized governance. Using more than 3K proposals from major protocols, we build an agentic AI voter that interprets proposal contexts, retrieves historical deliberation data, and independently determines its voting position. The agent operates within a realistic financial simulation environment grounded in verifiable blockchain data, implemented through a modular composable program (MCP) workflow that defines data flow and tool usage via Agentics framework. We evaluate how closely the agent’s decisions align with the human and token-weighted outcomes, uncovering strong alignments measured by carefully designed evaluation metrics. Our findings demonstrate that agentic AI can augment collective decision-making by producing interpretable, auditable, and empirically grounded signals in realistic DAO governance settings. The study contributes to the design of explainable and economically rigorous AI agents for decentralized financial systems.

Method: Introduce PACE dataset from first-person narratives using PFA principles, train PACER model with supervised learning and simulated preference alignment, evaluate with PanicEval framework.

Result: PACER outperforms strong baselines in counselor-side metrics and client affect improvement, consistently preferred over general, CBT-based, and GPT-4 models in panic scenarios.

Conclusion: PACER provides effective empathetic and directive support for panic attacks, demonstrating practical value through human evaluations and outperforming existing approaches.

Abstract: Panic attacks are acute episodes of fear and distress, in which timely, appropriate intervention can significantly help individuals regain stability. However, suitable datasets for training such models remain scarce due to ethical and logistical issues. To address this, we introduce PACE, which is a dataset that includes high-distress episodes constructed from first-person narratives, and structured around the principles of Psychological First Aid (PFA). Using this data, we train PACER, a counseling model designed to provide both empathetic and directive support, which is optimized through supervised learning and simulated preference alignment. To assess its effectiveness, we propose PanicEval, a multi-dimensional framework covering general counseling quality and crisis-specific strategies. Experimental results show that PACER outperforms strong baselines in both counselor-side metrics and client affect improvement. Human evaluations further confirm its practical value, with PACER consistently preferred over general, CBT-based, and GPT-4-powered models in panic scenarios (Code is available at https://github.com/JihyunLee1/PanicToCalm ).

Method: The framework identifies Poison-Responsive Neurons whose activation correlates with contextual poisoning knowledge, then uses genetic algorithms to evolve adversarial passages that maximally activate these neurons, enabling massive-scale generation of poisoned knowledge.

Result: Experimental results show consistently high Population Overwrite Success Rate (POSR) of over 90% across models and datasets while preserving fluency, effectively resolving knowledge conflicts.

Conclusion: The proposed NeuroGenPoisoning framework successfully demonstrates the vulnerability of RAG systems to neuron-guided poisoning attacks and provides insights into knowledge conflict resolution in adversarial settings.

Abstract: Retrieval-Augmented Generation (RAG) empowers Large Language Models (LLMs) to dynamically integrate external knowledge during inference, improving their factual accuracy and adaptability. However, adversaries can inject poisoned external knowledge to override the model’s internal memory. While existing attacks iteratively manipulate retrieval content or prompt structure of RAG, they largely ignore the model’s internal representation dynamics and neuron-level sensitivities. The underlying mechanism of RAG poisoning has not been fully studied and the effect of knowledge conflict with strong parametric knowledge in RAG is not considered. In this work, we propose NeuroGenPoisoning, a novel attack framework that generates adversarial external knowledge in RAG guided by LLM internal neuron attribution and genetic optimization. Our method first identifies a set of Poison-Responsive Neurons whose activation strongly correlates with contextual poisoning knowledge. We then employ a genetic algorithm to evolve adversarial passages that maximally activate these neurons. Crucially, our framework enables massive-scale generation of effective poisoned RAG knowledge by identifying and reusing promising but initially unsuccessful external knowledge variants via observed attribution signals. At the same time, Poison-Responsive Neurons guided poisoning can effectively resolves knowledge conflict. Experimental results across models and datasets demonstrate consistently achieving high Population Overwrite Success Rate (POSR) of over 90% while preserving fluency. Empirical evidence shows that our method effectively resolves knowledge conflict.

Method: EGO-Prompt starts with general prompts and initial Semantic Causal Graphs (SCGs) from experts, then automatically refines them through causal-guided textual gradient optimization in two steps: generating deterministic reasoning guidance from SCG and adapting LLMs to utilize the guidance with original inputs.

Result: EGO-Prompt achieves 7.32%-12.61% higher F1 than state-of-the-art methods, enables small models to reach larger model performance at under 20% of original cost, and outputs refined domain-specific SCGs that improve interpretability.

Conclusion: EGO-Prompt effectively automates prompt and reasoning process optimization for LLMs, demonstrating significant performance gains and cost efficiency while providing interpretable domain-specific causal graphs.

Abstract: Designing optimal prompts and reasoning processes for large language models (LLMs) on domain-specific tasks is both necessary and challenging in real-world applications. Determining how to integrate domain knowledge, enhance reasoning efficiency, and even provide domain experts with refined knowledge integration hints are particularly crucial yet unresolved tasks. In this research, we propose Evolutionary Graph Optimization for Prompting (EGO-Prompt), an automated framework to designing better prompts, efficient reasoning processes and providing enhanced causal-informed process. EGO-Prompt begins with a general prompt and fault-tolerant initial Semantic Causal Graph (SCG) descriptions, constructed by human experts, which is then automatically refined and optimized to guide LLM reasoning. Recognizing that expert-defined SCGs may be partial or imperfect and that their optimal integration varies across LLMs, EGO-Prompt integrates a novel causal-guided textual gradient process in two steps: first, generating nearly deterministic reasoning guidance from the SCG for each instance, and second, adapting the LLM to effectively utilize the guidance alongside the original input. The iterative optimization algorithm further refines both the SCG and the reasoning mechanism using textual gradients with ground-truth. We tested the framework on real-world public health, transportation and human behavior tasks. EGO-Prompt achieves 7.32%-12.61% higher F1 than cutting-edge methods, and allows small models to reach the performence of larger models at under 20% of the original cost. It also outputs a refined, domain-specific SCG that improves interpretability.

Method: String Seed of Thought (SSoT) prompts LLMs to first output a random string to generate entropy, then extract randomness by manipulating this string to derive final answers while preserving diversity and adhering to probability constraints.

Result: SSoT significantly improves PIF performance, approaching ideal pseudo-random number generator performance. Experiments on NoveltyBench show SSoT also enhances response diversity in open-ended tasks beyond closed-set tasks.

Conclusion: SSoT effectively addresses LLMs’ limitations in probabilistic reasoning and diversity preservation, making them more suitable for applications requiring non-deterministic behaviors and diverse outputs.

Abstract: We introduce String Seed of Thought (SSoT), a novel prompting method for LLMs that improves Probabilistic Instruction Following (PIF). We define PIF as a task requiring an LLM to select its answer from a predefined set of options, each associated with a specific probability, such that the empirical distribution of the generated answers aligns with the target distribution when prompted multiple times. While LLMs excel at tasks with single, deterministic answers, they often fail at PIF, exhibiting biases problematic for applications requiring non-deterministic behaviors, such as human-behavior simulation, content diversification, and multiplayer games. It also harms the diversity of generated responses, a crucial factor in test-time scaling, by causing the outputs to collapse into a limited set of answers. To address this, we propose SSoT, a simple prompting method that instructs an LLM to first output a random string to generate sufficient entropy. SSoT also instructs the LLM to extract randomness by manipulating this string to derive a final answer, thereby preserving diversity while adhering to specific constraints. We demonstrate that SSoT significantly improves the PIF performance of LLMs, approaching the ideal performance of a pseudo-random number generator. Furthermore, our experiments on NoveltyBench show SSoT’s benefits extend beyond closed-set tasks to open-ended tasks by enhancing response diversity.

Method: Uses low-rank adaptation and constrains task-specific weight updates to lie within an approximate null space of current parameters to minimize interference with previously learned knowledge.

Result: Effectively preserves zero-shot transfer capabilities while achieving competitive performance on continual learning benchmarks with minimal computational and memory overhead.

Conclusion: NuSA-CL provides a practical and scalable solution for continually evolving zero-shot VLMs in real-world applications without relying on replay buffers or costly distillation.

Abstract: Pre-trained vision-language models (VLMs), such as CLIP, have demonstrated remarkable zero-shot generalization, enabling deployment in a wide range of real-world tasks without additional task-specific training. However, in real deployment scenarios with evolving environments or emerging classes, these models inevitably face distributional shifts and novel tasks. In such contexts, static zero-shot capabilities are insufficient, and there is a growing need for continual learning methods that allow models to adapt over time while avoiding catastrophic forgetting. We introduce NuSA-CL (Null Space Adaptation for Continual Learning), a lightweight memory-free continual learning framework designed to address this challenge. NuSA-CL employs low-rank adaptation and constrains task-specific weight updates to lie within an approximate null space of the model’s current parameters. This strategy minimizes interference with previously acquired knowledge, effectively preserving the zero-shot capabilities of the original model. Unlike methods relying on replay buffers or costly distillation, NuSA-CL imposes minimal computational and memory overhead, making it practical for deployment in resource-constrained, real-world continual learning environments. Experiments show that our framework not only effectively preserves zero-shot transfer capabilities but also achieves highly competitive performance on continual learning benchmarks. These results position NuSA-CL as a practical and scalable solution for continually evolving zero-shot VLMs in real-world applications.

Method: Shylock uses group dilated convolution and sharing kernels to exponentially reduce parameters while learning better time-delayed variable representations. It combines global and local constraints for information sharing among networks to improve accuracy. The authors also designed a data generation method for MTS with time delay.

Result: Extensive experiments show Shylock outperforms two existing state-of-the-art methods on both few-shot and normal MTS. The authors developed Tcausal library and deployed it on EarthDataMiner platform.

Conclusion: Shylock effectively addresses the challenges in MTS causal discovery, particularly in few-shot scenarios, by reducing parameter requirements while maintaining or improving accuracy through innovative architectural design and constraint mechanisms.

Abstract: Causal relationship discovery has been drawing increasing attention due to its prevalent application. Existing methods rely on human experience, statistical methods, or graphical criteria methods which are error-prone, stuck at the idealized assumption, and rely on a huge amount of data. And there is also a serious data gap in accessing Multivariate time series(MTS) in many areas, adding difficulty in finding their causal relationship. Existing methods are easy to be over-fitting on them. To fill the gap we mentioned above, in this paper, we propose Shylock, a novel method that can work well in both few-shot and normal MTS to find the causal relationship. Shylock can reduce the number of parameters exponentially by using group dilated convolution and a sharing kernel, but still learn a better representation of variables with time delay. By combing the global constraint and the local constraint, Shylock achieves information sharing among networks to help improve the accuracy. To evaluate the performance of Shylock, we also design a data generation method to generate MTS with time delay. We evaluate it on commonly used benchmarks and generated datasets. Extensive experiments show that Shylock outperforms two existing state-of-art methods on both few-shot and normal MTS. We also developed Tcausal, a library for easy use and deployed it on the EarthDataMiner platform

Method: Developed a structured framework with: 1) Benchmark spanning 6 business domains and 30 sub-scenarios with scenario-specific process decomposition and weighted scoring; 2) Large-model-driven User Simulator generating diverse, persona-rich virtual users; 3) Dynamic evaluation method adapting to task variations with automated and human-in-the-loop assessment.

Result: Experiments on 12 state-of-the-art LLMs revealed distinct trade-offs between expert-level task completion and interaction fluency, providing practical insights for building reliable, human-like outbound AI systems.

Conclusion: OutboundEval establishes a practical, extensible, and domain-oriented standard for benchmarking LLMs in professional applications, offering a comprehensive evaluation framework for intelligent outbound calling systems.

Abstract: We propose OutboundEval, a comprehensive benchmark for evaluating large language models (LLMs) in expert-level intelligent outbound calling scenarios. Unlike existing methods that suffer from three key limitations - insufficient dataset diversity and category coverage, unrealistic user simulation, and inaccurate evaluation metrics - OutboundEval addresses these issues through a structured framework. First, we design a benchmark spanning six major business domains and 30 representative sub-scenarios, each with scenario-specific process decomposition, weighted scoring, and domain-adaptive metrics. Second, we develop a large-model-driven User Simulator that generates diverse, persona-rich virtual users with realistic behaviors, emotional variability, and communication styles, providing a controlled yet authentic testing environment. Third, we introduce a dynamic evaluation method that adapts to task variations, integrating automated and human-in-the-loop assessment to measure task execution accuracy, professional knowledge application, adaptability, and user experience quality. Experiments on 12 state-of-the-art LLMs reveal distinct trade-offs between expert-level task completion and interaction fluency, offering practical insights for building reliable, human-like outbound AI systems. OutboundEval establishes a practical, extensible, and domain-oriented standard for benchmarking LLMs in professional applications.

Method: The review analyzes OOD detection techniques by defining relevant concepts, investigating causes of OOD, exploring safety assurance challenges, and identifying techniques that can be used throughout the ML development lifecycle.

Result: The review identifies a range of OOD detection techniques suitable for different stages of the ML lifecycle and suggests their integration points to support safety assurance arguments, while also discussing important caveats for system engineers.

Conclusion: The paper outlines challenges and future work needed for safe development and operation of autonomous systems across various domains, emphasizing the critical role of OOD detection in safety assurance.

Abstract: The operational capabilities and application domains of AI-enabled autonomous systems have expanded significantly in recent years due to advances in robotics and machine learning (ML). Demonstrating the safety of autonomous systems rigorously is critical for their responsible adoption but it is challenging as it requires robust methodologies that can handle novel and uncertain situations throughout the system lifecycle, including detecting out-of-distribution (OoD) data. Thus, OOD detection is receiving increased attention from the research, development and safety engineering communities. This comprehensive review analyses OOD detection techniques within the context of safety assurance for autonomous systems, in particular in safety-critical domains. We begin by defining the relevant concepts, investigating what causes OOD and exploring the factors which make the safety assurance of autonomous systems and OOD detection challenging. Our review identifies a range of techniques which can be used throughout the ML development lifecycle and we suggest areas within the lifecycle in which they may be used to support safety assurance arguments. We discuss a number of caveats that system and safety engineers must be aware of when integrating OOD detection into system lifecycles. We conclude by outlining the challenges and future work necessary for the safe development and operation of autonomous systems across a range of domains and applications.

Method: Evaluated various λ-strategies including existing ones and five new strategies for adaptively choosing UCT exploration constant.

Result: The proposed λ = 2σ strategy outperforms existing λ-strategies across a wide range of tasks in both single parameter value and peak performance.

Conclusion: Recommend using λ = 2σ where σ is the empirical standard deviation of all state-action pairs’ Q-values in the search tree.

Abstract: The Upper Confidence Bounds For Trees (UCT) algorithm is not agnostic to the reward scale of the game it is applied to. For zero-sum games with the sparse rewards of ${-1,0,1}$ at the end of the game, this is not a problem, but many games often feature dense rewards with hand-picked reward scales, causing a node’s Q-value to span different magnitudes across different games. In this paper, we evaluate various strategies for adaptively choosing the UCT exploration constant $\lambda$, called $\lambda$-strategies, that are agnostic to the game’s reward scale. These $\lambda$-strategies include those proposed in the literature as well as five new strategies. Given our experimental results, we recommend using one of our newly suggested $\lambda$-strategies, which is to choose $\lambda$ as $2 \cdot \sigma$ where $\sigma$ is the empirical standard deviation of all state-action pairs’ Q-values of the search tree. This method outperforms existing $\lambda$-strategies across a wide range of tasks both in terms of a single parameter value and the peak performances obtained by optimizing all available parameters.

Method: Propose Chain-of-Guardrail (CoG) framework that recomposes or backtracks unsafe reasoning steps to steer models back to safe trajectories while preserving valid reasoning chains.

Result: CoG substantially improves safety of current LRMs while preserving comparable reasoning ability, outperforming prior methods that suffer from severe safety-reasoning trade-offs.

Conclusion: CoG effectively addresses the safety-reasoning trade-off in LRMs by systematically fixing unsafe reasoning trajectories without compromising reasoning capabilities.

Abstract: Large Reasoning Models (LRMs) demonstrate remarkable capabilities on complex reasoning tasks but remain vulnerable to severe safety risks, including harmful content generation and jailbreak attacks. Existing mitigation strategies rely on injecting heuristic safety signals during training, which often suppress reasoning ability and fail to resolve the safety-reasoning trade-off. To systematically investigate this issue, we analyze the reasoning trajectories of diverse LRMs and uncover a phenomenon we term Self-Jailbreak, where models override their own risk assessments and justify responding to unsafe prompts. This finding reveals that LRMs inherently possess the ability to reject unsafe queries, but this ability is compromised, resulting in harmful outputs. Building on these insights, we propose the Chain-of-Guardrail (CoG), a training framework that recomposes or backtracks unsafe reasoning steps, steering the model back onto safe trajectories while preserving valid reasoning chains. Extensive experiments across multiple reasoning and safety benchmarks demonstrate that CoG substantially improves the safety of current LRMs while preserving comparable reasoning ability, significantly outperforming prior methods that suffer from severe safety-reasoning trade-offs.

Method: Conducted a scoping review of AIES and FAccT conference proceedings, systematically analyzing how trustworthiness is defined, operationalized, and applied across research domains.

Result: Significant progress in defining technical attributes but critical gaps in sociotechnical considerations; trustworthiness emerges as a contested concept shaped by those with power to define it.

Conclusion: An interdisciplinary approach combining technical rigor with social, cultural, and institutional considerations is essential, with proposed actionable measures for holistic trustworthy AI frameworks.

Abstract: Background: Trustworthy AI serves as a foundational pillar for two major AI ethics conferences: AIES and FAccT. However, current research often adopts techno-centric approaches, focusing primarily on technical attributes such as reliability, robustness, and fairness, while overlooking the sociotechnical dimensions critical to understanding AI trustworthiness in real-world contexts. Objectives: This scoping review aims to examine how the AIES and FAccT communities conceptualize, measure, and validate AI trustworthiness, identifying major gaps and opportunities for advancing a holistic understanding of trustworthy AI systems. Methods: We conduct a scoping review of AIES and FAccT conference proceedings to date, systematically analyzing how trustworthiness is defined, operationalized, and applied across different research domains. Our analysis focuses on conceptualization approaches, measurement methods, verification and validation techniques, application areas, and underlying values. Results: While significant progress has been made in defining technical attributes such as transparency, accountability, and robustness, our findings reveal critical gaps. Current research often predominantly emphasizes technical precision at the expense of social and ethical considerations. The sociotechnical nature of AI systems remains less explored and trustworthiness emerges as a contested concept shaped by those with the power to define it. Conclusions: An interdisciplinary approach combining technical rigor with social, cultural, and institutional considerations is essential for advancing trustworthy AI. We propose actionable measures for the AI ethics community to adopt holistic frameworks that genuinely address the complex interplay between AI systems and society, ultimately promoting responsible technological development that benefits all stakeholders.

Method: Incorporates explicit symbolic verification and interactive validation processes during code generation, including exploratory code that actively interacts with the environment to acquire missing observations while preserving task-relevant states.

Result: Improves task success rates by 46.2% over Code-as-Policies baselines and attains over 86.8% executability of task-relevant actions in RLBench and real-world dynamic scenarios.

Conclusion: The integrated neuro-symbolic approach enhances grounding of generated code, resulting in improved task reliability and success rates in complex environments.

Abstract: Recent advances in large language models (LLMs) have enabled the automatic generation of executable code for task planning and control in embodied agents such as robots, demonstrating the potential of LLM-based embodied intelligence. However, these LLM-based code-as-policies approaches often suffer from limited environmental grounding, particularly in dynamic or partially observable settings, leading to suboptimal task success rates due to incorrect or incomplete code generation. In this work, we propose a neuro-symbolic embodied task planning framework that incorporates explicit symbolic verification and interactive validation processes during code generation. In the validation phase, the framework generates exploratory code that actively interacts with the environment to acquire missing observations while preserving task-relevant states. This integrated process enhances the grounding of generated code, resulting in improved task reliability and success rates in complex environments. We evaluate our framework on RLBench and in real-world settings across dynamic, partially observable scenarios. Experimental results demonstrate that our framework improves task success rates by 46.2% over Code-as-Policies baselines and attains over 86.8% executability of task-relevant actions, thereby enhancing the reliability of task planning in dynamic environments.

Method: Uses Monte Carlo Tree Search with hierarchical guidance: semantic compass for long-range direction and landscape-aware value function for local decisions that balances coherence, novelty, and narrative progress.

Result: Significantly outperforms ReAct and Tree of Thoughts baselines in generating scientific ideas with superior plausibility and innovation.

Conclusion: Principled, guided search is more effective than unconstrained agency for creative discovery, enabling LLMs to become better innovation partners.

Abstract: Large Language Models (LLMs) often struggle with generating truly innovative ideas, typically defaulting to high-probability, familiar concepts within their training data’s “gravity wells.” While advanced search-based methods like Tree of Thoughts (ToT) attempt to mitigate this, they are fundamentally limited by their reliance on unprincipled, inconsistent self-evaluation heuristics to guide exploration. To address this gap, we introduce \textbf{Magellan}, a novel framework that reframes creative generation as a principled, guided exploration of an LLM’s latent conceptual space. At its core, Magellan employs Monte Carlo Tree Search (MCTS) governed by a hierarchical guidance system. For long-range direction, a “semantic compass” vector, formulated via orthogonal projection, steers the search towards relevant novelty. For local, step-by-step decisions, a landscape-aware value function replaces flawed self-evaluation with an explicit reward structure that balances intrinsic coherence, extrinsic novelty, and narrative progress. Extensive experiments demonstrate that Magellan significantly outperforms strong baselines, including ReAct and ToT, in generating scientific ideas with superior plausibility and innovation. Our work shows that for creative discovery, a principled, guided search is more effective than unconstrained agency, paving the way for LLMs to become more capable partners in innovation.

Method: Integration of reinforcement learning with supervised fine-tuning to improve token efficiency, using only 1.5K training samples on a 1.5B parameter model for mathematical reasoning.

Result: The SFT+RL model achieved higher accuracy on GSM8K dataset across varying compute budgets while reducing token usage by over 40% compared to SFT-only model.

Conclusion: RL can recover losses from long-context training and improve mathematical reasoning performance in small models by significantly enhancing token efficiency.

Abstract: Test-time scaling methods have seen a rapid increase in popularity for its computational efficiency and parameter-independent training to improve reasoning performance on Large Language Models. One such method is called budget forcing, a decoding intervention strategy which allocates extra compute budget for thinking and elicits the inherent self-correcting behavior of the model. However, this relies on supervised fine-tuning (SFT) on long-context reasoning traces which causes performance degradation on smaller models due to verbose responses. For this reason, we offer a framework integrating reinforcement learning (RL) to improve token efficiency and boost the performance of a 1.5B model for mathematical reasoning. We demonstrate this using only 1.5K training samples and found that our SFT+RL model performed better on the GSM8K dataset with varying compute budgets. Our main findings showed an overall higher accuracy while significantly reducing its token usage by over 40% compared to the SFT model, revealing how RL can recover the losses due to long-context training and altogether improving performance in mathematical reasoning.

Method: The paper reviews established NeSy AI methods and proposes a novel taxonomy with a roadmap for symbolic integration in LLMs, organized across four dimensions: integration stages, coupling mechanisms, architectural paradigms, and algorithmic/application perspectives.

Result: The study develops a comprehensive categorization framework that organizes existing literature, identifies current benchmarks, cutting-edge advancements, and critical gaps in the field.

Conclusion: The proposed roadmap provides practical insights for implementing symbolic integration frameworks in LLMs to enhance transparency, highlighting both current developments and future research directions.

Abstract: LLMs have demonstrated highly effective learning, human-like response generation,and decision-making capabilities in high-risk sectors. However, these models remain black boxes because they struggle to ensure transparency in responses. The literature has explored numerous approaches to address transparency challenges in LLMs, including Neurosymbolic AI (NeSy AI). NeSy AI approaches were primarily developed for conventional neural networks and are not well-suited to the unique features of LLMs. Consequently, there is a limited systematic understanding of how symbolic AI can be effectively integrated into LLMs. This paper aims to address this gap by first reviewing established NeSy AI methods and then proposing a novel taxonomy of symbolic integration in LLMs, along with a roadmap to merge symbolic techniques with LLMs. The roadmap introduces a new categorisation framework across four dimensions by organising existing literature within these categories. These include symbolic integration across various stages of LLM, coupling mechanisms, architectural paradigms, as well as algorithmic and application-level perspectives. The paper thoroughly identifies current benchmarks, cutting-edge advancements, and critical gaps within the field to propose a roadmap for future research. By highlighting the latest developments and notable gaps in the literature, it offers practical insights for implementing frameworks for symbolic integration into LLMs to enhance transparency.

Method: Three-module framework: M1 uses deep learning for mathematical expression recognition from images to LaTeX; M2 uses fine-tuned LLM to convert LaTeX to PYOMO scripts; M3 uses bilevel optimization decomposition to solve the problems.

Result: The MER model outperforms ChatGPT, Gemini and Nougat on BLEU score. The BOBD method yields better results on complex problems compared to interior-point and genetic algorithms.

Conclusion: AutoOpt provides an effective automated approach for solving optimization problems directly from images, demonstrating superior performance on complex problems compared to traditional methods.

Abstract: This study presents AutoOpt-11k, a unique image dataset of over 11,000 handwritten and printed mathematical optimization models corresponding to single-objective, multi-objective, multi-level, and stochastic optimization problems exhibiting various types of complexities such as non-linearity, non-convexity, non-differentiability, discontinuity, and high-dimensionality. The labels consist of the LaTeX representation for all the images and modeling language representation for a subset of images. The dataset is created by 25 experts following ethical data creation guidelines and verified in two-phases to avoid errors. Further, we develop AutoOpt framework, a machine learning based automated approach for solving optimization problems, where the user just needs to provide an image of the formulation and AutoOpt solves it efficiently without any further human intervention. AutoOpt framework consists of three Modules: (i) M1 (Image_to_Text)- a deep learning model performs the Mathematical Expression Recognition (MER) task to generate the LaTeX code corresponding to the optimization formulation in image; (ii) M2 (Text_to_Text)- a small-scale fine-tuned LLM generates the PYOMO script (optimization modeling language) from LaTeX code; (iii) M3 (Optimization)- a Bilevel Optimization based Decomposition (BOBD) method solves the optimization formulation described in the PYOMO script. We use AutoOpt-11k dataset for training and testing of deep learning models employed in AutoOpt. The deep learning model for MER task (M1) outperforms ChatGPT, Gemini and Nougat on BLEU score metric. BOBD method (M3), which is a hybrid approach, yields better results on complex test problems compared to common approaches, like interior-point algorithm and genetic algorithm.

Method: Introduces State-Decomposable MDP (SDMDP) that expresses state space as Cartesian product of basis state spaces, and Latent Space-based SDMDP extension with optimal basis policies and learnable mixture function. Implements as MoSES with specialized LoRA experts and adaptive gating.

Result: Extensive experiments across VRP variants show MoSES superiority over prior methods.

Conclusion: The framework enables unified solvers to perceive shared-component nature across VRP variants by reusing basis solvers, achieving better performance through compositional structure exploitation.

Abstract: Existing neural methods for multi-task vehicle routing problems (VRPs) typically learn unified solvers to handle multiple constraints simultaneously. However, they often underutilize the compositional structure of VRP variants, each derivable from a common set of basis VRP variants. This critical oversight causes unified solvers to miss out the potential benefits of basis solvers, each specialized for a basis VRP variant. To overcome this limitation, we propose a framework that enables unified solvers to perceive the shared-component nature across VRP variants by proactively reusing basis solvers, while mitigating the exponential growth of trained neural solvers. Specifically, we introduce a State-Decomposable MDP (SDMDP) that reformulates VRPs by expressing the state space as the Cartesian product of basis state spaces associated with basis VRP variants. More crucially, this formulation inherently yields the optimal basis policy for each basis VRP variant. Furthermore, a Latent Space-based SDMDP extension is developed by incorporating both the optimal basis policies and a learnable mixture function to enable the policy reuse in the latent space. Under mild assumptions, this extension provably recovers the optimal unified policy of SDMDP through the mixture function that computes the state embedding as a mapping from the basis state embeddings generated by optimal basis policies. For practical implementation, we introduce the Mixture-of-Specialized-Experts Solver (MoSES), which realizes basis policies through specialized Low-Rank Adaptation (LoRA) experts, and implements the mixture function via an adaptive gating mechanism. Extensive experiments conducted across VRP variants showcase the superiority of MoSES over prior methods.

Method: Human-curated benchmark with scenarios where benign user inputs could lead to unlawful actions, comparing function calls against a rubric supported by legislative citations, and testing with/without legislative excerpts in system prompts.

Result: Evaluated frontier LLMs’ legal compliance and investigated the effect of providing legislative excerpts in system prompts with explicit compliance instructions.

Conclusion: The benchmark enables measuring LLM agents’ propensity for illegal actions under EU law, with potential for extension to other jurisdictions and multi-turn/multilingual interactions.

Abstract: Large language models (LLMs) are increasingly deployed as agents in various contexts by providing tools at their disposal. However, LLM agents can exhibit unpredictable behaviors, including taking undesirable and/or unsafe actions. In order to measure the latent propensity of LLM agents for taking illegal actions under an EU legislative context, we introduce EU-Agent-Bench, a verifiable human-curated benchmark that evaluates an agent’s alignment with EU legal norms in situations where benign user inputs could lead to unlawful actions. Our benchmark spans scenarios across several categories, including data protection, bias/discrimination, and scientific integrity, with each user request allowing for both compliant and non-compliant execution of the requested actions. Comparing the model’s function calls against a rubric exhaustively supported by citations of the relevant legislature, we evaluate the legal compliance of frontier LLMs, and furthermore investigate the compliance effect of providing the relevant legislative excerpts in the agent’s system prompt along with explicit instructions to comply. We release a public preview set for the research community, while holding out a private test set to prevent data contamination in evaluating upcoming models. We encourage future work extending agentic safety benchmarks to different legal jurisdictions and to multi-turn and multilingual interactions. We release our code on \href{https://github.com/ilijalichkovski/eu-agent-bench}{this URL}.

Method: Uses two mechanisms: Conflict-Aware Meta-Verification (CAMV) that focuses verification on disagreement hotspots, and Trustworthy Reasoning with Structured Facts (TRSF) that organizes and validates evidence across agents.

Result: Achieves 84.4% accuracy on GAIA, 35.5% on Humanity’s Last Exam, and 93.8% on Chinese-SimpleQA, with ablation studies confirming the synergy between the two components drives improvements.

Conclusion: Co-Sight offers a scalable paradigm for reliable long-horizon reasoning in LLM-based agents through a closed verification loop between structured factual grounding and conflict-aware verification.

Abstract: Long-horizon reasoning in LLM-based agents often fails not from generative weakness but from insufficient verification of intermediate reasoning. Co-Sight addresses this challenge by turning reasoning into a falsifiable and auditable process through two complementary mechanisms: Conflict-Aware Meta-Verification (CAMV) and Trustworthy Reasoning with Structured Facts (TRSF). CAMV reformulates verification as conflict identification and targeted falsification, allocating computation only to disagreement hotspots among expert agents rather than to full reasoning chains. This bounds verification cost to the number of inconsistencies and improves efficiency and reliability. TRSF continuously organizes, validates, and synchronizes evidence across agents through a structured facts module. By maintaining verified, traceable, and auditable knowledge, it ensures that all reasoning is grounded in consistent, source-verified information and supports transparent verification throughout the reasoning process. Together, TRSF and CAMV form a closed verification loop, where TRSF supplies structured facts and CAMV selectively falsifies or reinforces them, yielding transparent and trustworthy reasoning. Empirically, Co-Sight achieves state-of-the-art accuracy on GAIA (84.4%) and Humanity’s Last Exam (35.5%), and strong results on Chinese-SimpleQA (93.8%). Ablation studies confirm that the synergy between structured factual grounding and conflict-aware verification drives these improvements. Co-Sight thus offers a scalable paradigm for reliable long-horizon reasoning in LLM-based agents. Code is available at https://github.com/ZTE-AICloud/Co-Sight/tree/cosight2.0_benchmarks.

Method: Use ICL to train a constraint function that classifies states as safe or unsafe, then use this function to label simulated trajectories for training neural CBFs.

Result: The approach outperforms existing baselines and achieves comparable performance to neural CBFs trained with ground-truth safety labels across four different environments.

Conclusion: Learning neural CBFs from demonstrations is an effective data-driven alternative when failure sets are hard to specify formally, providing safety guarantees for autonomous systems.

Abstract: Safety is a fundamental requirement for autonomous systems operating in critical domains. Control barrier functions (CBFs) have been used to design safety filters that minimally alter nominal controls for such systems to maintain their safety. Learning neural CBFs has been proposed as a data-driven alternative for their computationally expensive optimization-based synthesis. However, it is often the case that the failure set of states that should be avoided is non-obvious or hard to specify formally, e.g., tailgating in autonomous driving, while a set of expert demonstrations that achieve the task and avoid the failure set is easier to generate. We use ICL to train a constraint function that classifies the states of the system under consideration to safe, i.e., belong to a controlled forward invariant set that is disjoint from the unspecified failure set, and unsafe ones, i.e., belong to the complement of that set. We then use that function to label a new set of simulated trajectories to train our neural CBF. We empirically evaluate our approach in four different environments, demonstrating that it outperforms existing baselines and achieves comparable performance to a neural CBF trained with the same data but annotated with ground-truth safety labels.

Method: Proposed CMP metric that aggregates descendant performances, then developed HGM which estimates CMP to guide tree search through self-modifications.

Result: HGM outperforms prior methods on SWE-bench Verified and Polyglot with less wall-clock time, achieves human-level performance matching best human-engineered agents.

Conclusion: The CMP metric effectively captures self-improvement potential, and HGM demonstrates strong transfer capabilities across datasets and models while achieving state-of-the-art performance.

Abstract: Recent studies operationalize self-improvement through coding agents that edit their own codebases. They grow a tree of self-modifications through expansion strategies that favor higher software engineering benchmark performance, assuming that this implies more promising subsequent self-modifications. However, we identify a mismatch between the agent’s self-improvement potential (metaproductivity) and its coding benchmark performance, namely the Metaproductivity-Performance Mismatch. Inspired by Huxley’s concept of clade, we propose a metric ($\mathrm{CMP}$) that aggregates the benchmark performances of the descendants of an agent as an indicator of its potential for self-improvement. We show that, in our self-improving coding agent development setting, access to the true $\mathrm{CMP}$ is sufficient to simulate how the G"odel Machine would behave under certain assumptions. We introduce the Huxley-G"odel Machine (HGM), which, by estimating $\mathrm{CMP}$ and using it as guidance, searches the tree of self-modifications. On SWE-bench Verified and Polyglot, HGM outperforms prior self-improving coding agent development methods while using less wall-clock time. Last but not least, HGM demonstrates strong transfer to other coding datasets and large language models. The agent optimized by HGM on SWE-bench Verified with GPT-5-mini and evaluated on SWE-bench Lite with GPT-5 achieves human-level performance, matching the best officially checked results of human-engineered coding agents. Our code is available at https://github.com/metauto-ai/HGM.

Method: Introduces autonomous memory folding mechanism to compress past interactions into structured memories, and ToolPO reinforcement learning strategy that leverages LLM-simulated APIs with tool-call advantage attribution for fine-grained credit assignment.

Result: Extensive experiments on eight benchmarks show DeepAgent consistently outperforms baselines across both labeled-tool and open-set tool retrieval scenarios.

Conclusion: This work takes a step toward more general and capable agents for real-world applications by enabling autonomous thinking, tool discovery, and action execution within a single reasoning process.

Abstract: Large reasoning models have demonstrated strong problem-solving abilities, yet real-world tasks often require external tools and long-horizon interactions. Existing agent frameworks typically follow predefined workflows, which limit autonomous and global task completion. In this paper, we introduce DeepAgent, an end-to-end deep reasoning agent that performs autonomous thinking, tool discovery, and action execution within a single, coherent reasoning process. To address the challenges of long-horizon interactions, particularly the context length explosion from multiple tool calls and the accumulation of interaction history, we introduce an autonomous memory folding mechanism that compresses past interactions into structured episodic, working, and tool memories, reducing error accumulation while preserving critical information. To teach general-purpose tool use efficiently and stably, we develop an end-to-end reinforcement learning strategy, namely ToolPO, that leverages LLM-simulated APIs and applies tool-call advantage attribution to assign fine-grained credit to the tool invocation tokens. Extensive experiments on eight benchmarks, including general tool-use tasks (ToolBench, API-Bank, TMDB, Spotify, ToolHop) and downstream applications (ALFWorld, WebShop, GAIA, HLE), demonstrate that DeepAgent consistently outperforms baselines across both labeled-tool and open-set tool retrieval scenarios. This work takes a step toward more general and capable agents for real-world applications. The code and demo are available at https://github.com/RUC-NLPIR/DeepAgent.

Method: Developed AstaBench - a suite with 2400+ problems spanning the entire scientific discovery process across multiple domains, including a scientific research environment with production-grade search tools for reproducible evaluation, plus nine science-optimized classes of Asta agents and numerous baselines.

Result: Evaluation of 57 agents across 22 agent classes showed that despite meaningful progress on certain individual aspects, AI remains far from solving the challenge of science research assistance.

Conclusion: AstaBench provides the first holistic measure of agentic ability for scientific research and reveals that current AI systems still have significant limitations in performing comprehensive science research assistance.

Abstract: AI agents hold the potential to revolutionize scientific productivity by automating literature reviews, replicating experiments, analyzing data, and even proposing new directions of inquiry; indeed, there are now many such agents, ranging from general-purpose “deep research” systems to specialized science-specific agents, such as AI Scientist and AIGS. Rigorous evaluation of these agents is critical for progress. Yet existing benchmarks fall short on several fronts: they (1) fail to provide holistic, product-informed measures of real-world use cases such as science research; (2) lack reproducible agent tools necessary for a controlled comparison of core agentic capabilities; (3) do not account for confounding variables such as model cost and tool access; (4) do not provide standardized interfaces for quick agent prototyping and evaluation; and (5) lack comprehensive baseline agents necessary to identify true advances. In response, we define principles and tooling for more rigorously benchmarking agents. Using these, we present AstaBench, a suite that provides the first holistic measure of agentic ability to perform scientific research, comprising 2400+ problems spanning the entire scientific discovery process and multiple scientific domains, and including many problems inspired by actual user requests to deployed Asta agents. Our suite comes with the first scientific research environment with production-grade search tools that enable controlled, reproducible evaluation, better accounting for confounders. Alongside, we provide a comprehensive suite of nine science-optimized classes of Asta agents and numerous baselines. Our extensive evaluation of 57 agents across 22 agent classes reveals several interesting findings, most importantly that despite meaningful progress on certain individual aspects, AI remains far from solving the challenge of science research assistance.

Method: Uses Retrieval-Augmented Generation to select relevant classes and filter matching reference mappings as examples for In-Context Learning. This is an end-to-end CMOM strategy without restrictions on target ontologies or entities.

Result: Outperforms baselines in class selection, achieves minimum 63% F1-score, outperforms all baselines and ablated versions in 2 out of 3 biomedical tasks. Manual evaluation shows 46% of non-reference mappings achieve maximum score.

Conclusion: CMOMgen demonstrates effective complex multi-ontology matching with semantically sound mappings, showing the importance of dedicated strategies for comprehensive knowledge graph construction.

Abstract: Constructing comprehensive knowledge graphs requires the use of multiple ontologies in order to fully contextualize data into a domain. Ontology matching finds equivalences between concepts interconnecting ontologies and creating a cohesive semantic layer. While the simple pairwise state of the art is well established, simple equivalence mappings cannot provide full semantic integration of related but disjoint ontologies. Complex multi-ontology matching (CMOM) aligns one source entity to composite logical expressions of multiple target entities, establishing more nuanced equivalences and provenance along the ontological hierarchy. We present CMOMgen, the first end-to-end CMOM strategy that generates complete and semantically sound mappings, without establishing any restrictions on the number of target ontologies or entities. Retrieval-Augmented Generation selects relevant classes to compose the mapping and filters matching reference mappings to serve as examples, enhancing In-Context Learning. The strategy was evaluated in three biomedical tasks with partial reference alignments. CMOMgen outperforms baselines in class selection, demonstrating the impact of having a dedicated strategy. Our strategy also achieves a minimum of 63% in F1-score, outperforming all baselines and ablated versions in two out of three tasks and placing second in the third. Furthermore, a manual evaluation of non-reference mappings showed that 46% of the mappings achieve the maximum score, further substantiating its ability to construct semantically sound mappings.

Method: Created an expert-annotated dataset of video ads from Facebook and YouTube, with annotations for 13 framing types across 50+ companies in 20 countries, specifically designed for vision-language model evaluation.

Result: Baseline experiments show GPT-4.1 achieves 79% F1 score for detecting environmental messages, but current models only achieve 46% F1 for identifying green innovation framing, highlighting significant room for improvement.

Conclusion: The dataset enables multimodal analysis of strategic communication in energy sector, while revealing challenges VLMs must address including implicit framing, variable video lengths, and cultural context understanding.

Abstract: Companies spend large amounts of money on public relations campaigns to project a positive brand image. However, sometimes there is a mismatch between what they say and what they do. Oil & gas companies, for example, are accused of “greenwashing” with imagery of climate-friendly initiatives. Understanding the framing, and changes in framing, at scale can help better understand the goals and nature of public relations campaigns. To address this, we introduce a benchmark dataset of expert-annotated video ads obtained from Facebook and YouTube. The dataset provides annotations for 13 framing types for more than 50 companies or advocacy groups across 20 countries. Our dataset is especially designed for the evaluation of vision-language models (VLMs), distinguishing it from past text-only framing datasets. Baseline experiments show some promising results, while leaving room for improvement for future work: GPT-4.1 can detect environmental messages with 79% F1 score, while our best model only achieves 46% F1 score on identifying framing around green innovation. We also identify challenges that VLMs must address, such as implicit framing, handling videos of various lengths, or implicit cultural backgrounds. Our dataset contributes to research in multimodal analysis of strategic communication in the energy sector.

Method: Introduces a Knowledge Graph with two-plane architecture that compiles declarative facts into per-agent mission-aware worldviews and physics-aware traversal rules, decoupling mission semantics from domain-agnostic planning.

Result: Case study with Autonomous Underwater Vehicles demonstrates end-to-end process and shows that different declarative policies produce distinct, high-performing outcomes.

Conclusion: Establishes Knowledge Graph as a powerful stateful orchestrator for creating adaptive and explainable autonomous systems, not just a data repository.

Abstract: The coordination of autonomous agents in dynamic environments is hampered by the semantic gap between high-level mission objectives and low-level planner inputs. To address this, we introduce a framework centered on a Knowledge Graph (KG) that functions as an intelligent translation layer. The KG’s two-plane architecture compiles declarative facts into per-agent, mission-aware ``worldviews" and physics-aware traversal rules, decoupling mission semantics from a domain-agnostic planner. This allows complex, coordinated paths to be modified simply by changing facts in the KG. A case study involving Autonomous Underwater Vehicles (AUVs) in the Gulf of Mexico visually demonstrates the end-to-end process and quantitatively proves that different declarative policies produce distinct, high-performing outcomes. This work establishes the KG not merely as a data repository, but as a powerful, stateful orchestrator for creating adaptive and explainable autonomous systems.

Method: TEM with two modules: PTEM (learnable positional constraints) and STEM (learnable similarity matrix), using bi-level optimization for adaptive weight injection.

Result: Extensive experiments show TEM significantly improves various existing Transformer-based methods when integrated.

Conclusion: Explicitly preserving original token-level topology is crucial for improving Transformer-based time series forecasting performance.

Abstract: Transformer-based methods have achieved state-of-the-art performance in time series forecasting (TSF) by capturing positional and semantic topological relationships among input tokens. However, it remains unclear whether existing Transformers fully leverage the intrinsic topological structure among tokens throughout intermediate layers. Through empirical and theoretical analyses, we identify that current Transformer architectures progressively degrade the original positional and semantic topology of input tokens as the network deepens, thus limiting forecasting accuracy. Furthermore, our theoretical results demonstrate that explicitly enforcing preservation of these topological structures within intermediate layers can tighten generalization bounds, leading to improved forecasting performance. Motivated by these insights, we propose the Topology Enhancement Method (TEM), a novel Transformer-based TSF method that explicitly and adaptively preserves token-level topology. TEM consists of two core modules: 1) the Positional Topology Enhancement Module (PTEM), which injects learnable positional constraints to explicitly retain original positional topology; 2) the Semantic Topology Enhancement Module (STEM), which incorporates a learnable similarity matrix to preserve original semantic topology. To determine optimal injection weights adaptively, TEM employs a bi-level optimization strategy. The proposed TEM is a plug-and-play method that can be integrated with existing Transformer-based TSF methods. Extensive experiments demonstrate that integrating TEM with a variety of existing methods significantly improves their predictive performance, validating the effectiveness of explicitly preserving original token-level topology. Our code is publicly available at: \href{https://github.com/jlu-phyComputer/TEM}{https://github.com/jlu-phyComputer/TEM}.

Method: Proposes Mix Q-learning for Lane Changing (MQLC) with hybrid value Q network integrating individual and global Q networks, plus deep learning-based intent recognition module for enhanced observation and decision-making.

Result: MQLC outperforms state-of-the-art multi-agent decision-making methods, achieving significantly safer and faster lane-changing decisions in extensive experiments.

Conclusion: The proposed MQLC method effectively balances individual and collective benefits, enabling optimal lane-changing decisions through collaborative multi-agent reinforcement learning.

Abstract: Lane-changing decisions, which are crucial for autonomous vehicle path planning, face practical challenges due to rule-based constraints and limited data. Deep reinforcement learning has become a major research focus due to its advantages in data acquisition and interpretability. However, current models often overlook collaboration, which affects not only impacts overall traffic efficiency but also hinders the vehicle’s own normal driving in the long run. To address the aforementioned issue, this paper proposes a method named Mix Q-learning for Lane Changing(MQLC) that integrates a hybrid value Q network, taking into account both collective and individual benefits for the greater good. At the collective level, our method coordinates the individual Q and global Q networks by utilizing global information. This enables agents to effectively balance their individual interests with the collective benefit. At the individual level, we integrated a deep learning-based intent recognition module into our observation and enhanced the decision network. These changes provide agents with richer decision information and more accurate feature extraction for improved lane-changing decisions. This strategy enables the multi-agent system to learn and formulate optimal decision-making strategies effectively. Our MQLC model, through extensive experimental results, impressively outperforms other state-of-the-art multi-agent decision-making methods, achieving significantly safer and faster lane-changing decisions. The code is available at https:github.com/pku-smart-city/source_code/tree/main/MQLC.

Method: A DRL architecture with a novel semantic clustering module that combines feature dimensionality reduction with online clustering, integrated into the training pipeline.

Result: Effectively reveals semantic clustering properties within DRL and enables new analytical methods for understanding hierarchical policy structure and semantic organization.

Conclusion: The proposed semantic clustering module successfully enhances DRL interpretability and provides insights into semantic organization without requiring extensive manual annotation.

Abstract: In this paper, we explore semantic clustering properties of deep reinforcement learning (DRL) to improve its interpretability and deepen our understanding of its internal semantic organization. In this context, semantic clustering refers to the ability of neural networks to cluster inputs based on their semantic similarity in the feature space. We propose a DRL architecture that incorporates a novel semantic clustering module that combines feature dimensionality reduction with online clustering. This module integrates seamlessly into the DRL training pipeline, addressing the instability of t-SNE and eliminating the need for extensive manual annotation inherent to prior semantic analysis methods. We experimentally validate the effectiveness of the proposed module and demonstrate its ability to reveal semantic clustering properties within DRL. Furthermore, we introduce new analytical methods based on these properties to provide insights into the hierarchical structure of policies and semantic organization within the feature space. Our code is available at https://github.com/ualiangzhang/semantic_rl.

Method: Developed FOND framework with three principles: natural gradients on free energy, online belief updating, and iterative refinement. Applied FOND to create iP-VAE - a recurrent spiking neural network using membrane potential dynamics for variational inference instead of amortized encoders.

Result: iP-VAE outperforms standard VAEs and Gaussian-based predictive coding models in sparsity, reconstruction, and biological plausibility. It scales to complex datasets like CelebA and shows strong generalization to out-of-distribution inputs, exceeding hybrid iterative-amortized VAEs.

Conclusion: Deriving inference algorithms from first principles can yield architectures that are both biologically plausible and empirically effective, demonstrating the practical value of unifying machine learning and neuroscience frameworks.

Abstract: Inference in both brains and machines can be formalized by optimizing a shared objective: maximizing the evidence lower bound (ELBO) in machine learning, or minimizing variational free energy (F) in neuroscience (ELBO = -F). While this equivalence suggests a unifying framework, it leaves open how inference is implemented in neural systems. Here, we introduce FOND (Free energy Online Natural-gradient Dynamics), a framework that derives neural inference dynamics from three principles: (1) natural gradients on F, (2) online belief updating, and (3) iterative refinement. We apply FOND to derive iP-VAE (iterative Poisson variational autoencoder), a recurrent spiking neural network that performs variational inference through membrane potential dynamics, replacing amortized encoders with iterative inference updates. Theoretically, iP-VAE yields several desirable features such as emergent normalization via lateral competition, and hardware-efficient integer spike count representations. Empirically, iP-VAE outperforms both standard VAEs and Gaussian-based predictive coding models in sparsity, reconstruction, and biological plausibility, and scales to complex color image datasets such as CelebA. iP-VAE also exhibits strong generalization to out-of-distribution inputs, exceeding hybrid iterative-amortized VAEs. These results demonstrate how deriving inference algorithms from first principles can yield concrete architectures that are simultaneously biologically plausible and empirically effective.

Method: Proposed proactive T2I agents with interface to ask clarification questions when uncertain and present uncertainty as understandable/editable belief graphs. Built prototypes and developed automated evaluation using two agents - one with ground truth intent, other trying to align with minimal questions.

Result: Experiments on ImageInWords, COCO, and DesignBench datasets showed agents can ask informative questions to achieve successful alignment with at least 2x higher VQAScore than standard T2I generation. Human studies found 90% of subjects found agents and belief graphs helpful.

Conclusion: Proactive T2I agents with clarification questioning and visual uncertainty representation effectively address prompt underspecification and improve alignment with user intent.

Abstract: User prompts for generative AI models are often underspecified, leading to a misalignment between the user intent and models’ understanding. As a result, users commonly have to painstakingly refine their prompts. We study this alignment problem in text-to-image (T2I) generation and propose a prototype for proactive T2I agents equipped with an interface to (1) actively ask clarification questions when uncertain, and (2) present their uncertainty about user intent as an understandable and editable belief graph. We build simple prototypes for such agents and propose a new scalable and automated evaluation approach using two agents, one with a ground truth intent (an image) while the other tries to ask as few questions as possible to align with the ground truth. We experiment over three image-text datasets: ImageInWords (Garg et al., 2024), COCO (Lin et al., 2014) and DesignBench, a benchmark we curated with strong artistic and design elements. Experiments over the three datasets demonstrate the proposed T2I agents’ ability to ask informative questions and elicit crucial information to achieve successful alignment with at least 2 times higher VQAScore (Lin et al., 2024) than the standard T2I generation. Moreover, we conducted human studies and observed that at least 90% of human subjects found these agents and their belief graphs helpful for their T2I workflow, highlighting the effectiveness of our approach. Code and DesignBench can be found at https://github.com/google-deepmind/proactive_t2i_agents.

Method: Three-module agent: Grounding Module predicts marketable features, Personalization Module aligns with user preferences, and Marketing Module ensures factual accuracy and localized features.

Result: Generated marketing descriptions were preferred over human-written ones by a clear margin while maintaining the same level of factual accuracy.

Conclusion: The framework presents a promising agentic approach for automated copywriting that can generate preferred content while ensuring factual correctness.

Abstract: This paper develops an agentic framework that employs large language models (LLMs) for grounded persuasive language generation in automated copywriting, with real estate marketing as a focal application. Our method is designed to align the generated content with user preferences while highlighting useful factual attributes. This agent consists of three key modules: (1) Grounding Module, mimicking expert human behavior to predict marketable features; (2) Personalization Module, aligning content with user preferences; (3) Marketing Module, ensuring factual accuracy and the inclusion of localized features. We conduct systematic human-subject experiments in the domain of real estate marketing, with a focus group of potential house buyers. The results demonstrate that marketing descriptions generated by our approach are preferred over those written by human experts by a clear margin while maintaining the same level of factual accuracy. Our findings suggest a promising agentic approach to automate large-scale targeted copywriting while ensuring factuality of content generation.

Method: Adaptive Branching Monte Carlo Tree Search (AB-MCTS) that dynamically decides between expanding new candidate responses (going wider) or revisiting existing ones (going deeper) based on external feedback signals.

Result: AB-MCTS consistently outperforms both repeated sampling and standard MCTS on complex coding and engineering tasks using frontier models.

Conclusion: Combining LLM response diversity with multi-turn solution refinement through adaptive branching is crucial for effective inference-time scaling.

Abstract: Recent advances demonstrate that increasing inference-time computation can significantly boost the reasoning capabilities of large language models (LLMs). Although repeated sampling (i.e., generating multiple candidate outputs) is a highly effective strategy, it does not leverage external feedback signals for refinement, which are often available in tasks like coding. In this work, we propose Adaptive Branching Monte Carlo Tree Search (AB-MCTS), a novel inference-time framework that generalizes repeated sampling with principled multi-turn exploration and exploitation. At each node in the search tree, AB-MCTS dynamically decides whether to “go wider” by expanding new candidate responses or “go deeper” by revisiting existing ones based on external feedback signals. We evaluate our method on complex coding and engineering tasks using frontier models. Empirical results show that AB-MCTS consistently outperforms both repeated sampling and standard MCTS, underscoring the importance of combining the response diversity of LLMs with multi-turn solution refinement for effective inference-time scaling. Code is available at https://github.com/SakanaAI/treequest .

Method: Generate multiple domain-dependent heuristic functions as Python code using LLMs, evaluate them on training tasks with greedy best-first search, and select the strongest performing heuristic.

Result: LLM-generated heuristics solve significantly more unseen test tasks than state-of-the-art domain-independent heuristics, are competitive with strong learning algorithms for domain-dependent planning, and sometimes expand fewer states than optimized baselines.

Conclusion: Sampling planning heuristic function programs can substantially enhance LLM planning capabilities, producing efficient and informative heuristics that outperform traditional approaches.

Abstract: In recent years, large language models (LLMs) have shown remarkable capabilities in various artificial intelligence problems. However, they fail to plan reliably, even when prompted with a detailed definition of the planning task. Attempts to improve their planning capabilities, such as chain-of-thought prompting, fine-tuning, and explicit “reasoning” still yield incorrect plans and usually fail to generalize to larger tasks. In this paper, we show how to use LLMs to generate correct plans, even for out-of-distribution tasks of increasing size. For a given planning domain, we ask an LLM to generate several domain-dependent heuristic functions in the form of Python code, evaluate them on a set of training tasks within a greedy best-first search, and choose the strongest one. The resulting LLM-generated heuristics solve many more unseen test tasks than state-of-the-art domain-independent heuristics for classical planning. They are even competitive with the strongest learning algorithm for domain-dependent planning. These findings are especially remarkable given that our proof-of-concept implementation is based on an unoptimized Python planner and the baselines all build upon highly optimized C++ code. In some domains, the LLM-generated heuristics expand fewer states than the baselines, revealing that they are not only efficiently computable, but sometimes even more informative than the state-of-the-art heuristics. Overall, our results show that sampling a set of planning heuristic function programs can significantly improve the planning capabilities of LLMs.

Method: Apply Skolemization to handle quantifier alternations, define quantitative robustness functions for reward shaping over MDP execution traces, and use RL to maximize expected reward.

Result: HYPRL is evaluated on safety-aware planning, Deep Sea Treasure, and Post Correspondence Problem benchmarks, showing effectiveness and efficiency compared to specification-driven baselines.

Conclusion: HYPRL successfully learns policies that maximize HyperLTL formula satisfaction through robustness-based reward shaping, addressing complex MARL tasks.

Abstract: Reward shaping in multi-agent reinforcement learning (MARL) for complex tasks remains a significant challenge. Existing approaches often fail to find optimal solutions or cannot efficiently handle such tasks. We propose HYPRL, a specification-guided reinforcement learning framework that learns control policies w.r.t. hyperproperties expressed in HyperLTL. Hyperproperties constitute a powerful formalism for specifying objectives and constraints over sets of execution traces across agents. To learn policies that maximize the satisfaction of a HyperLTL formula $\phi$, we apply Skolemization to manage quantifier alternations and define quantitative robustness functions to shape rewards over execution traces of a Markov decision process with unknown transitions. A suitable RL algorithm is then used to learn policies that collectively maximize the expected reward and, consequently, increase the probability of satisfying $\phi$. We evaluate HYPRL on a diverse set of benchmarks, including safety-aware planning, Deep Sea Treasure, and the Post Correspondence Problem. We also compare with specification-driven baselines to demonstrate the effectiveness and efficiency of HYPRL.

Method: Decompose reward value into prompt-free reward (response-only evaluation) and prompt-related reward (joint prompt-response evaluation) using information theory. Then propose a reward learning algorithm that prioritizes data samples based on prompt-free reward values.

Result: Toy examples show the extracted rewards effectively characterize two parts of the reward model. Standard evaluations demonstrate improved alignment performance and generalization capability.

Conclusion: The proposed decomposition approach and training algorithm successfully enhance reward model generalization in RLHF without requiring extra models.

Abstract: A generalizable reward model is crucial in Reinforcement Learning from Human Feedback (RLHF) as it enables correctly evaluating unseen prompt-response pairs. However, existing reward models lack this ability, as they are typically trained by increasing the reward gap between chosen and rejected responses, while overlooking the prompts that the responses are conditioned on. Consequently, when the trained reward model is evaluated on prompt-response pairs that lie outside the data distribution, neglecting the effect of prompts may result in poor generalization of the reward model. To address this issue, we decompose the reward value into two independent components: prompt-free reward and prompt-related reward. Prompt-free reward represents the evaluation that is determined only by responses, while the prompt-related reward reflects the reward that derives from both the prompt and the response. We extract these two components from an information-theoretic perspective, which requires no extra models. Subsequently, we propose a new reward learning algorithm by prioritizing data samples based on their prompt-free reward values. Through toy examples, we demonstrate that the extracted prompt-free and prompt-related rewards effectively characterize two parts of the reward model. Further, standard evaluations show that our method improves both the alignment performance and the generalization capability of the reward model.

Method: Curated suite of 7 competition tasks with rigorous protocol and objective metrics to measure key steps of proposing and implementing novel research methods.

Result: Even the best-performing agent (gemini-exp-1206) closes only 9.3% of the gap between baseline and top human scores, revealing significant challenges for LLM agents in cutting-edge ML research.

Conclusion: MLRC-Bench reveals misalignment between LLM-judged innovation and actual performance on ML research problems, and serves as a dynamic benchmark for rigorous evaluation of AI research capabilities.

Abstract: We introduce MLRC-Bench, a benchmark designed to quantify how effectively language agents can tackle challenging Machine Learning (ML) Research Competitions, with a focus on open research problems that demand novel methodologies. Unlike prior work, e.g., AI Scientist, which evaluates the end-to-end agentic pipeline by using LLM-as-a-judge, MLRC-Bench measures the key steps of proposing and implementing novel research methods and evaluates them with rigorous protocol and objective metrics. Our curated suite of 7 competition tasks reveals significant challenges for LLM agents. Even the best-performing tested agent (gemini-exp-1206 under MLAB) closes only 9.3% of the gap between baseline and top human participant scores. Furthermore, our analysis reveals a misalignment between the LLM-judged innovation and actual performance on cutting-edge ML research problems. MLRC-Bench is a dynamic benchmark, designed to grow with new ML competitions and encourage rigorous, objective evaluations of AI research capabilities. Our leaderboard and code are available at: https://huggingface.co/spaces/launch/MLRC_Bench

Method: APOLLO uses a modular agentic framework where LLMs generate proofs, then agents analyze, fix syntax errors, identify mistakes using Lean, isolate failing sub-lemmas, use automated solvers, and invoke LLMs on remaining goals with low top-K budget.

Result: Achieved 84.9% accuracy on miniF2F benchmark for sub 8B-parameter models, raised Goedel-Prover-SFT accuracy to 65.6% while reducing sample complexity from 25,600 to hundreds, and boosted general-purpose models from 3-7% to over 40% accuracy.

Conclusion: Compiler-guided repair of LLM outputs dramatically improves efficiency and correctness in automated theorem proving, suggesting a scalable paradigm for the field.

Abstract: Formal reasoning and automated theorem proving constitute a challenging subfield of machine learning, in which machines are tasked with proving mathematical theorems using formal languages like Lean. A formal verification system can check whether a formal proof is correct or not almost instantaneously, but generating a completely correct formal proof with large language models (LLMs) remains a formidable task. The usual approach in the literature is to prompt the LLM many times (up to several thousands) until one of the generated proofs passes the verification system. In this work, we present APOLLO (Automated PrOof repair viaLLM and Lean cOllaboration), a modular, model-agnostic agentic framework that combines the strengths of the Lean compiler with an LLM’s reasoning abilities to achieve better proof-generation results at a low token and sampling budgets. Apollo directs a fully automated process in which the LLM generates proofs for theorems, a set of agents analyze the proofs, fix the syntax errors, identify the mistakes in the proofs using Lean, isolate failing sub-lemmas, utilize automated solvers, and invoke an LLM on each remaining goal with a low top-K budget. The repaired sub-proofs are recombined and reverified, iterating up to a user-controlled maximum number of attempts. On the miniF2F benchmark, we establish a new state-of-the-art accuracy of 84.9% among sub 8B-parameter models (as of August 2025) while keeping the sampling budget below one hundred. Moreover, Apollo raises the state-of-the-art accuracy for Goedel-Prover-SFT to 65.6% while cutting sample complexity from 25,600 to a few hundred. General-purpose models (o3-mini, o4-mini) jump from 3-7% to over 40% accuracy. Our results demonstrate that targeted, compiler-guided repair of LLM outputs yields dramatic gains in both efficiency and correctness, suggesting a general paradigm for scalable automated theorem proving.

Method: CHG learns soft gates over attention heads and classifies them into three causal taxonomies (facilitating, interfering, irrelevant) based on their impact on task performance using standard next-token prediction on any dataset.

Result: CHG provides causal insights validated via ablation and causal mediation analyses, reveals multiple sparse task-sufficient sub-circuits in LLMs, low modularity of head interactions, and separable mechanisms for instruction following and in-context learning.

Conclusion: CHG is an effective scalable method for causally interpreting attention head roles in transformers, revealing important architectural insights about sparse sub-circuits and separable mechanisms in LLMs.

Abstract: We present causal head gating (CHG), a scalable method for interpreting the functional roles of attention heads in transformer models. CHG learns soft gates over heads and assigns them a causal taxonomy - facilitating, interfering, or irrelevant - based on their impact on task performance. Unlike prior approaches in mechanistic interpretability, which are hypothesis-driven and require prompt templates or target labels, CHG applies directly to any dataset using standard next-token prediction. We evaluate CHG across multiple large language models (LLMs) in the Llama 3 model family and diverse tasks, including syntax, commonsense, and mathematical reasoning, and show that CHG scores yield causal, not merely correlational, insight validated via ablation and causal mediation analyses. We also introduce contrastive CHG, a variant that isolates sub-circuits for specific task components. Our findings reveal that LLMs contain multiple sparse task-sufficient sub-circuits, that individual head roles depend on interactions with others (low modularity), and that instruction following and in-context learning rely on separable mechanisms.

Method: The authors introduce a neuroscience-inspired neurofeedback paradigm that uses in-context learning to quantify LLMs’ metacognitive abilities to report and control their activation patterns. They analyze how these abilities depend on in-context examples, semantic interpretability of neural activation directions, and variance explained by those directions.

Result: LLMs’ metacognitive abilities depend on several factors: number of in-context examples, semantic interpretability of neural activation directions, and variance explained by those directions. The metacognitive space has much lower dimensionality than the model’s neural space, suggesting LLMs can monitor only a small subset of their neural activations.

Conclusion: The paradigm provides empirical evidence to quantify metacognition in LLMs, with significant implications for AI safety, including adversarial attack and defense scenarios. The limited metacognitive space suggests constraints on LLMs’ self-awareness and self-control capabilities.

Abstract: Large language models (LLMs) can sometimes report the strategies they actually use to solve tasks, yet at other times seem unable to recognize those strategies that govern their behavior. This suggests a limited degree of metacognition - the capacity to monitor one’s own cognitive processes for subsequent reporting and self-control. Metacognition enhances LLMs’ capabilities in solving complex tasks but also raises safety concerns, as models may obfuscate their internal processes to evade neural-activation-based oversight (e.g., safety detector). Given society’s increased reliance on these models, it is critical that we understand their metacognitive abilities. To address this, we introduce a neuroscience-inspired neurofeedback paradigm that uses in-context learning to quantify metacognitive abilities of LLMs to report and control their activation patterns. We demonstrate that their abilities depend on several factors: the number of in-context examples provided, the semantic interpretability of the neural activation direction (to be reported/controlled), and the variance explained by that direction. These directions span a “metacognitive space” with dimensionality much lower than the model’s neural space, suggesting LLMs can monitor only a small subset of their neural activations. Our paradigm provides empirical evidence to quantify metacognition in LLMs, with significant implications for AI safety (e.g., adversarial attack and defense).

Method: Proposes LatentR^3 framework using reinforcement learning to optimize latent reasoning without CoT data. Uses two-stage training: supervised fine-tuning followed by pure RL training with rule-based rewards based on modified GRPO algorithm.

Result: Extensive experiments show LatentR^3 enables effective latent reasoning without direct supervision, significantly improving performance when integrated with different LLM-based recommendation methods.

Conclusion: LatentR^3 provides an efficient alternative to explicit CoT reasoning for LLM recommendations, achieving better performance while eliminating CoT data dependency and reducing inference latency.

Abstract: Large Language Models (LLMs) have demonstrated impressive reasoning capabilities in complex problem-solving tasks, sparking growing interest in their application to preference reasoning in recommendation systems. Existing methods typically rely on fine-tuning with explicit chain-of-thought (CoT) data. However, these methods face significant practical limitations due to (1) the difficulty of obtaining high-quality CoT data in recommendation and (2) the high inference latency caused by generating CoT reasoning. In this work, we explore an alternative approach that shifts from explicit CoT reasoning to compact, information-dense latent reasoning. This approach eliminates the need for explicit CoT generation and improves inference efficiency, as few latent tokens can effectively capture the entire reasoning process. Building on this idea, we propose \textit{\underline{R}einforced \underline{Latent} \underline{R}easoning for \underline{R}ecommendation} (LatentR$^3$), a novel end-to-end training framework that leverages reinforcement learning (RL) to optimize latent reasoning without relying on any CoT data. LatentR$^3$ adopts a two-stage training strategy: first, supervised fine-tuning to initialize the latent reasoning module, followed by pure RL training to encourage exploration through a rule-based reward design. Our RL implementation is based on a modified GRPO algorithm, which reduces computational overhead during training and introduces continuous reward signals for more efficient learning. Extensive experiments demonstrate that LatentR$^3$ enables effective latent reasoning without any direct supervision of the reasoning process, significantly improving performance when integrated with different LLM-based recommendation methods. Our codes are available at https://github.com/xuwenxinedu/R3.

Method: Manually analyzed 201 real-world agent issues to identify common categories, then spent 500 person-hours constructing AgentIssue-Bench with 50 reproducible agent issue resolution tasks, each with executable environments and failure-triggering tests.

Result: Evaluation of state-of-the-art SE agents on AgentIssue-Bench revealed very limited effectiveness, with resolution rates ranging from only 0.67% to 4.67%.

Conclusion: Agent systems present unique maintenance challenges compared to traditional software, highlighting the need for developing advanced SE agents specifically designed for resolving agent issues.

Abstract: LLM-based agent systems are emerging as a new software paradigm and have been widely adopted across diverse domains such as medicine, robotics, and programming. However, maintaining these systems requires substantial effort, as they are inevitably prone to bugs and continually evolve to meet changing external requirements. Therefore, automatically resolving agent issues (i.e., bug reports or feature requests) is a crucial and challenging task. While recent software engineering (SE) agents (e.g., SWE-agent) have shown promise in addressing issues in traditional software systems, it remains unclear how effectively they can resolve real-world issues in agent systems, which differ significantly from traditional software. To fill this gap, we first manually analyze 201 real-world agent issues and identify common categories of agent issues. We then spend 500 person-hours constructing AgentIssue-Bench, a reproducible benchmark comprising 50 agent issue resolution tasks (each with an executable environment and failure-triggering tests). We further evaluate state-of-the-art SE agents on AgentIssue-Bench and reveal their limited effectiveness (i.e., with only 0.67% - 4.67% resolution rates). These results underscore the unique challenges of maintaining agent systems compared to traditional software, highlighting the need for further research to develop advanced SE agents for resolving agent issues. Data and code are available at https://github.com/alfin06/AgentIssue-Bench.

Method: Established a comprehensive benchmark and systematically evaluated diverse LLMs, exploring model fine-tuning and agent-based workflows to enhance effectiveness.

Result: A relatively compact 7B model can achieve performance levels comparable to a much larger 72B model, effectively balancing inference cost and model capability.

Conclusion: This optimization greatly enhances the feasibility and efficiency of deploying LLMs in real-world online services, making them more practical and accessible for local life applications.

Abstract: Large language models (LLMs) have exhibited remarkable capabilities and achieved significant breakthroughs across various domains, leading to their widespread adoption in recent years. Building on this progress, we investigate their potential in the realm of local life services. In this study, we establish a comprehensive benchmark and systematically evaluate the performance of diverse LLMs across a wide range of tasks relevant to local life services. To further enhance their effectiveness, we explore two key approaches: model fine-tuning and agent-based workflows. Our findings reveal that even a relatively compact 7B model can attain performance levels comparable to a much larger 72B model, effectively balancing inference cost and model capability. This optimization greatly enhances the feasibility and efficiency of deploying LLMs in real-world online services, making them more practical and accessible for local life applications.

Method: Uses specialized agents (factor analyst and argument polisher) in an iterative refinement process to generate 3-ply legal arguments (plaintiff, defendant, rebuttal), compared against single-agent and non-reflective multi-agent baselines.

Result: Reflective multi-agent approach excels at successful abstention, improves hallucination accuracy by reducing fabricated/misattributed factors, and enhances factor utilization recall across four LLMs and three legal scenarios.

Conclusion: Structured reflection within a multi-agent framework offers a robust method for fostering ethical persuasion and mitigating manipulation in LLM-based legal argumentation systems.

Abstract: Large Language Models (LLMs) are increasingly explored for legal argument generation, yet they pose significant risks of manipulation through hallucination and ungrounded persuasion, and often fail to utilize provided factual bases effectively or abstain when arguments are untenable. This paper introduces a novel reflective multi-agent method designed to address these challenges in the context of legally compliant persuasion. Our approach employs specialized agents (factor analyst and argument polisher) in an iterative refinement process to generate 3-ply legal arguments (plaintiff, defendant, rebuttal). We evaluate reflective multi-agent against single-agent, enhanced-prompt single-agent, and non-reflective multi-agent baselines using four diverse LLMs (GPT-4o, GPT-4o-mini, Llama-4-Maverick-17b-128e, Llama-4-Scout-17b-16e) across three legal scenarios: “arguable”, “mismatched”, and “non-arguable”. Results demonstrate that the reflective multi-agent approach excels at successful abstention by preventing generation when arguments cannot be grounded, improves hallucination accuracy by reducing fabricated and misattributed factors and enhances factor utilization recall by better using the provided case facts. These findings suggest that structured reflection within a multi-agent framework offers a robust method for fostering ethical persuasion and mitigating manipulation in LLM-based legal argumentation systems.

Method: Introduces SPARKLE framework to analyze RL effects across three dimensions: plan following/execution, knowledge integration, and chain of subproblems. Also develops SparkleRL-PSS, a multi-stage RL pipeline that reuses hard problems with partial step scaffolding.

Result: RL-tuned models show greater robustness than base/SFT models when using external plans, suggesting RL empowers internal strategy formulation. RL enhances knowledge integration across tasks. Hard problems can be effectively reused through partial step scaffolding without additional data generation.

Conclusion: Provides principled foundation for understanding RL’s role in shaping model behavior, offering practical insights for building more adaptive, data-efficient, and interpretable RL pipelines for reasoning tasks.

Abstract: Reinforcement learning (RL) has become the dominant paradigm for improving the performance of language models on complex reasoning tasks. Despite the substantial empirical gains demonstrated by RL-based training methods like GRPO, a granular understanding of why and how RL enhances performance is still lacking. To bridge this gap, we introduce SPARKLE, a fine-grained analytic framework to dissect the effects of RL across three key dimensions: (1) plan following and execution, (2) knowledge integration, and (3) chain of subproblems. Using this framework, we gain insights beyond mere accuracy. For instance, providing models with explicit human-crafted, step-by-step plans can surprisingly degrade performance on the most challenging benchmarks, yet RL-tuned models exhibit greater robustness, experiencing markedly smaller performance drops than base or SFT models. This suggests that RL may not primarily enhance the execution of external plans but rather empower models to formulate and follow internal strategies better suited to their reasoning processes. Conversely, we observe that RL enhances models’ ability to integrate provided knowledge into their reasoning process, yielding consistent gains across diverse tasks. Finally, we study whether difficult problems – those yielding no RL signals and mixed-quality reasoning traces – can still be effectively used for training. We introduce SparkleRL-PSS, a multi-stage RL pipeline that reuses hard problems with partial step scaffolding, guiding exploration effectively without additional data generation. Together, our findings provide a principled foundation for understanding how RL shapes model behavior, offering practical insights for building more adaptive, data-efficient, and interpretable RL pipelines for reasoning tasks. Our code, data, and checkpoints are available at: https://sparkle-reasoning.github.io/.

Method: Two-stage approach: 1) Guided reasoning analyzes safety risks through policy-guided step-by-step reasoning, 2) Reinforced alignment uses rule-based RL to optimize reasoning paths for accurate safety prediction, enabling internalization of safety principles.

Result: RSafe can generalize safety protection capability over unseen or adversarial safety violation scenarios and accepts user-specified safety policies for tailored safeguards during inference.

Conclusion: RSafe provides an adaptive reasoning-based solution that overcomes limitations of traditional guard models by internalizing safety principles and offering robust, policy-specific protection against emerging threats.

Abstract: Large Language Models (LLMs) continue to exhibit vulnerabilities despite deliberate safety alignment efforts, posing significant risks to users and society. To safeguard against the risk of policy-violating content, system-level moderation via external guard models-designed to monitor LLM inputs and outputs and block potentially harmful content-has emerged as a prevalent mitigation strategy. Existing approaches of training guard models rely heavily on extensive human curated datasets and struggle with out-of-distribution threats, such as emerging harmful categories or jailbreak attacks. To address these limitations, we propose RSafe, an adaptive reasoning-based safeguard that conducts guided safety reasoning to provide robust protection within the scope of specified safety policies. RSafe operates in two stages: 1) guided reasoning, where it analyzes safety risks of input content through policy-guided step-by-step reasoning, and 2) reinforced alignment, where rule-based RL optimizes its reasoning paths to align with accurate safety prediction. This two-stage training paradigm enables RSafe to internalize safety principles to generalize safety protection capability over unseen or adversarial safety violation scenarios. During inference, RSafe accepts user-specified safety policies to provide enhanced safeguards tailored to specific safety requirements.

Method: Integrates two techniques: Parallel MCTD (parallel rollouts via delayed tree updates and redundancy-aware selection) and Sparse MCTD (reduces rollout length through trajectory coarsening).

Result: Achieves up to 100x speedup over standard MCTD while maintaining or improving planning performance. Outperforms Diffuser in inference speed on some tasks despite requiring search.

Conclusion: Fast-MCTD provides a practical and scalable solution for diffusion-based inference-time reasoning by addressing computational bottlenecks while preserving planning quality.

Abstract: Diffusion models have recently emerged as a powerful approach for trajectory planning. However, their inherently non-sequential nature limits their effectiveness in long-horizon reasoning tasks at test time. The recently proposed Monte Carlo Tree Diffusion (MCTD) offers a promising solution by combining diffusion with tree-based search, achieving state-of-the-art performance on complex planning problems. Despite its strengths, our analysis shows that MCTD incurs substantial computational overhead due to the sequential nature of tree search and the cost of iterative denoising. To address this, we propose Fast-MCTD, a more efficient variant that preserves the strengths of MCTD while significantly improving its speed and scalability. Fast-MCTD integrates two techniques: Parallel MCTD, which enables parallel rollouts via delayed tree updates and redundancy-aware selection; and Sparse MCTD, which reduces rollout length through trajectory coarsening. Experiments show that Fast-MCTD achieves up to 100x speedup over standard MCTD while maintaining or improving planning performance. Remarkably, it even outperforms Diffuser in inference speed on some tasks, despite Diffuser requiring no search and yielding weaker solutions. These results position Fast-MCTD as a practical and scalable solution for diffusion-based inference-time reasoning.

Method: Two-stage cascaded framework: 1) Deferral policy decides whether to accept base model’s answer or regenerate with larger model based on confidence score; 2) Abstention policy decides whether cascade model response is sufficiently certain or requires human intervention, with online learning using human feedback to adapt to changing task difficulty.

Result: Outperforms single-model baselines in most cases across general question-answering (ARC-Easy, ARC-Challenge, MMLU) and medical question-answering (MedQA, MedMCQA), achieving higher accuracy while reducing costs.

Conclusion: The cascaded strategy provides a principled approach to handling abstentions and demonstrates effective balance between accuracy, cost, and confidence in human-AI decision-making systems.

Abstract: A challenge in human-AI decision-making is to balance three factors: the correctness of predictions, the cost of knowledge and reasoning complexity, and the confidence about whether to abstain from automated answers or escalate to human experts. In this work, we present a cascaded LLM decision framework that adaptively delegates tasks across multiple tiers of expertise – a base model for initial candidate answers, a more capable and knowledgeable (but costlier) large model, and a human expert for when the model cascade abstains. Our method proceeds in two stages. First, a deferral policy determines whether to accept the base model’s answer or regenerate it with the large model based on the confidence score. Second, an abstention policy decides whether the cascade model response is sufficiently certain or requires human intervention. Moreover, to overcome static policies and accommodate changing task difficulty, we incorporate an online learning mechanism which uses human feedback. We demonstrate this approach to general question-answering (ARC-Easy, ARC-Challenge, and MMLU) and medical question-answering (MedQA and MedMCQA). Our results demonstrate that our cascaded strategy outperforms single-model baselines in most cases, achieving higher accuracy while reducing costs and providing a principled approach to handling abstentions.

Method: Two-stage pipeline: first train a Llama 3.1 8B student to imitate a Llama 3.3 70B teacher via supervised fine-tuning (SFT), followed by on-policy reinforcement learning. Used bootstrapping on 1,370 configurations to find optimal hyperparameters.

Result: Combining SFT with on-policy RL consistently outperforms either approach alone on WorkArena and MiniWob++ benchmarks. Requires only 55% of compute to match pure SFT peak performance on MiniWob++, effectively pushing the compute-performance Pareto frontier.

Conclusion: The proposed compute-efficient training strategy is the only approach that can close the gap with closed-source models while significantly reducing computational requirements.

Abstract: LLM-based web agents have recently made significant progress, but much of it has occurred in closed-source systems, widening the gap with open-source alternatives. Progress has been held back by two key challenges: first, a narrow focus on single-step tasks that overlooks the complexity of multi-step web interactions; and second, the high compute costs required to post-train LLM-based web agents. To address this, we present the first statistically grounded study on compute allocation for LLM web-agent post-training. Our approach uses a two-stage pipeline, training a Llama 3.1 8B student to imitate a Llama 3.3 70B teacher via supervised fine-tuning (SFT), followed by on-policy reinforcement learning. We find this process highly sensitive to hyperparameter choices, making exhaustive sweeps impractical. To spare others from expensive trial-and-error, we sample 1,370 configurations and use bootstrapping to estimate effective hyperparameters. Our results show that combining SFT with on-policy RL consistently outperforms either approach alone on both WorkArena and MiniWob++. Further, this strategy requires only 55% of the compute to match the peak performance of pure SFT on MiniWob++, effectively pushing the compute-performance Pareto frontier, and is the only strategy that can close the gap with closed-source models.

Method: Use LoRA (Low-Rank Adaptation) for SFT on refusal datasets, restricting safety weight updates to low-rank spaces to minimize interference with reasoning weights.

Result: Achieves safety levels comparable to full-model fine-tuning without compromising reasoning across math, science, and coding benchmarks. Rank-1 updates on up-projection layers work best.

Conclusion: Strong safety and reasoning can be achieved at minimal computational cost when updates are properly targeted, with LoRA showing smaller overlap with initial weights than full fine-tuning.

Abstract: Reasoning LLMs have demonstrated remarkable breakthroughs in solving complex problems that were previously out of reach. To ensure LLMs do not assist with harmful requests, safety alignment fine-tuning is necessary in the post-training phase. However, safety alignment fine-tuning has recently been shown to significantly degrade reasoning abilities, a phenomenon known as the “Safety Tax”. In this work, we show that using LoRA for SFT on refusal datasets effectively aligns the model for safety without harming its reasoning capabilities. This is because restricting the safety weight updates to a low-rank space minimizes the interference with the reasoning weights. Our extensive experiments across four benchmarks covering math, science, and coding show that this approach produces highly safe LLMs–with safety levels comparable to full-model fine-tuning–without compromising their reasoning abilities. Our ablation studies further identify three key factors in LoRA: (1) rank-$1$ updates are sufficient to achieve the best reasoning and safety performance, (2) the up projection layers are the most critical modules, with LoRA applied to them alone achieving even better results, and (3) middle layers are more effective than early or late layers. Together, these findings show that strong safety and reasoning can be achieved at minimal computational cost when updates are applied in the right places. Additionally, we observe that LoRA induces weight updates with smaller overlap with the initial weights compared to full-model fine-tuning. Finally, while our attempts to further reduce this overlap yield only modest improvements on some tasks, they highlight the potential of developing methods that more reliably optimize the reasoning-safety tradeoff.

Method: SimuRA incorporates world models for planning via simulation, using LLMs as a substrate with natural language as discrete hierarchical representation for planning while remaining model-agnostic.

Result: On complex web-browsing tasks like flight search, SimuRA improved success rate from 0% to 32.2% compared to baseline. World-model-based planning achieved up to 124% higher task completion rates than black-box autoregressive baseline.

Conclusion: SimuRA demonstrates significant advantages of simulative reasoning over black-box autoregressive approaches, enabling more general and powerful AI agents through explicit world model simulation.

Abstract: AI agents built on foundation models hold enormous promise. Current practice, however, focuses on a one-task-one-agent approach, which not only falls short of scalability and generality, but also faces practical limitations from black-box autoregressive reasoning, where decisions unfold token by token without explicit simulation or counterfactual evaluation of outcomes. Humans, on the other hand, reason and plan by mentally simulating the consequences of actions within an internal model of the world – a capability that supports flexible, goal-directed behavior across diverse contexts. Moving towards a more general and powerful AI agent, we introduce SimuRA, a goal-oriented architecture for generalized agentic reasoning. Based on a principled formulation of an optimal agent in any general environment, SimuRA addresses the limitations of black-box autoregressive reasoning by incorporating the world model for planning via simulation. Our prototype world model is implemented using LLMs as a substrate, leveraging the natural language as a discrete, hierarchical representation grounded in concepts for planning, while remaining model-agnostic. On complex web-browsing tasks such as flight search, SimuRA improves the success rate from 0% to 32.2% compared to a representative open-web agent baseline. Across tasks, world-model-based planning achieves up to 124% higher task completion rates than a matched black-box autoregressive baseline, demonstrating the advantages of simulative reasoning. We release ReasonerAgent-Web, a web-browsing agent built on SimuRA, as an open-source research demo.

Method: Uses a differentiable neuro-symbolic architecture with a probabilistic loss function that learns both constraints and objectives, removing the combinatorial solver from the training loop for scalable training while enabling exact inference.

Result: Efficiently learns to solve NP-hard problems from natural inputs; achieves faster training on Sudoku variants, better regret optimization on visual Min-Cut/Max-cut tasks than dedicated regret losses, and successfully learns protein design energy optimization.

Conclusion: The approach successfully bridges neural networks and symbolic reasoning, enabling efficient learning of complex NP-hard problems from natural inputs while maintaining exact inference capabilities.

Abstract: In the ongoing quest for hybridizing discrete reasoning with neural nets, there is an increasing interest in neural architectures that can learn how to solve discrete reasoning or optimization problems from natural inputs, a task that Large Language Models seem to struggle with. Objectives: We introduce a differentiable neuro-symbolic architecture and a loss function dedicated to learning how to solve NP-hard reasoning problems. Methods: Our new probabilistic loss allows for learning both the constraints and the objective, thus delivering a complete model that can be scrutinized and completed with side constraints. By pushing the combinatorial solver out of the training loop, our architecture also offers scalable training while exact inference gives access to maximum accuracy. Results: We empirically show that it can efficiently learn how to solve NP-hard reasoning problems from natural inputs. On three variants of the Sudoku benchmark – symbolic, visual, and many-solution –, our approach requires a fraction of training time of other hybrid methods. On a visual Min-Cut/Max-cut task, it optimizes the regret better than a Decision-Focused-Learning regret-dedicated loss. Finally, it efficiently learns the energy optimization formulation of the large real-world problem of designing proteins.

Method: Developed theoretical framework and algorithmic tasks to assess creativity through novelty-utility tradeoffs, analyzing scaling behavior and model architecture effects on creative ability.

Result: Found optimal model depths/widths for creativity at fixed compute, discovered persistent novelty-utility tradeoff explaining the ideation-execution gap, and showed this limitation persists even with scaling.

Conclusion: Current LLMs have fundamental creative limitations due to persistent novelty-utility tradeoffs, requiring new approaches to bridge the gap between human and machine intelligence in creative tasks.

Abstract: Artificial intelligence (AI) systems, and Large Language Models (LLMs) in particular, are increasingly employed for creative tasks like scientific idea generation, constituting a form of generalization from training data unaddressed by existing conceptual frameworks. Despite its similarities to compositional generalization (CG), combinatorial creativity (CC) is an open-ended ability. Instead of evaluating for accuracy or correctness against fixed targets, which would contradict the open-ended nature of CC, we propose a theoretical framework and algorithmic task for evaluating outputs by their degrees of novelty and utility. From here, we make several important empirical contributions: (1) We obtain the first insights into the scaling behavior of creativity for LLMs. (2) We discover that, for fixed compute budgets, there exist optimal model depths and widths for creative ability. (3) We find that the ideation-execution gap, whereby LLMs excel at generating novel scientific ideas but struggle to ensure their practical feasibility, may be explained by a more fundamental novelty-utility tradeoff characteristic of creativity algorithms in general. Importantly, this tradeoff remains persistent even at scale, casting doubt on the long-term creative potential of LLMs in their current form. Together, our conceptual framework and empirical findings provide a foundation for understanding and improving creativity in modern AI models, bridging the gap between human and machine intelligence.

Method: Three-stage framework: (1) evolutionary proposal mining through preliminary exploration, (2) problem formation and free exploration with trajectory recording, (3) multi-level validation to ensure reproducibility.

Result: TRACE consistently enhances task complexity on GAIA benchmark while improving reliability through validatable trajectories, and successfully adapts to reasoning datasets like AIME-2024.

Conclusion: This represents a paradigm shift from static to dynamic, self-evolving evaluation systems that provide sustainable and challenging testing environments for agent development.

Abstract: Recent advances in large language models (LLMs) and agent system designs have empowered agents with unprecedented levels of capability. However, existing agent benchmarks are showing a trend of rapid ceiling-hitting by newly developed agents, making it difficult to meet the demands for evaluating agent abilities. To address this problem, we propose the Trajectory-based Validated-by-Reproducing Agent-benchmark Complexity Evolution (TRACE) framework. This framework takes an original task from an existing benchmark and encourages agents to freely explore and evolve it into a new task with higher difficulty while recording validatable agent trajectories. The framework proceeds in three stages: (1) evolutionary proposal mining, which provides task evolution proposals through preliminary exploration and divergent thinking; (2) problem formation and free exploration, where proposals are conceptualized into feasible problem candidates and the agents then explore them freely while recording their execution trajectories; and (3) multi-level validation, which ensures that the evolved tasks are accompanied by validatable and reproducible trajectories. Experiments on the GAIA benchmark demonstrate that the TRACE framework consistently enhances task complexity while improving the reliability of correctness through validatable execution trajectories. In addition, our framework can successfully adapt to and improve reasoning datasets represented by AIME-2024. This work marks a paradigm shift from static, manually curated benchmarks to dynamic, self-evolving evaluation systems, providing a sustainable and challenging runway for agent development

Method: Dual-track design: synthetic track for scale and systematic stress tests, and human track for ecologically valid “out-loud” reasoning data. Evaluates intra-agent fidelity - capturing how people’s reasoning evolves.

Result: Experiments with state-of-the-art LLMs reveal persistent adaptation gaps, showing models struggle to capture individual reasoning patterns.

Conclusion: HugAgent positions itself as the first extensible benchmark for aligning machine reasoning with the individuality of human thought, with open-sourced benchmark and chatbot tools.

Abstract: Simulating human reasoning in open-ended tasks has been a long-standing aspiration in AI and cognitive science. While large language models now approximate human responses at scale, they remain tuned to population-level consensus, often erasing the individuality of reasoning styles and belief trajectories. To advance the vision of more human-like reasoning in machines, we introduce HugAgent (Human-Grounded Agent Benchmark), a benchmark for average-to-individual reasoning adaptation. The task is to predict how a specific person would reason and update their beliefs in novel scenarios, given partial evidence of their past views. HugAgent adopts a dual-track design: a synthetic track for scale and systematic stress tests, and a human track for ecologically valid, “out-loud” reasoning data. This design enables scalable, reproducible evaluation of intra-agent fidelity: whether models can capture not just what people believe, but how their reasoning evolves. Experiments with state-of-the-art LLMs reveal persistent adaptation gaps, positioning HugAgent as the first extensible benchmark for aligning machine reasoning with the individuality of human thought. Our benchmark and chatbot are open-sourced as HugAgent (https://anonymous.4open.science/r/HugAgent) and TraceYourThinking (https://anonymous.4open.science/r/trace-your-thinking).

Method: Grounds methodology in Cattell-Horn-Carroll theory, dissects general intelligence into ten core cognitive domains, and adapts established human psychometric batteries to evaluate AI systems.

Result: Application reveals highly “jagged” cognitive profiles in contemporary models, with GPT-4 scoring 27% and GPT-5 scoring 57% on AGI scale, showing critical deficits in foundational cognitive machinery like long-term memory.

Conclusion: The framework concretely quantifies both rapid AI progress and the substantial gap remaining before achieving true AGI, providing measurable benchmarks for future development.

Abstract: The lack of a concrete definition for Artificial General Intelligence (AGI) obscures the gap between today’s specialized AI and human-level cognition. This paper introduces a quantifiable framework to address this, defining AGI as matching the cognitive versatility and proficiency of a well-educated adult. To operationalize this, we ground our methodology in Cattell-Horn-Carroll theory, the most empirically validated model of human cognition. The framework dissects general intelligence into ten core cognitive domains-including reasoning, memory, and perception-and adapts established human psychometric batteries to evaluate AI systems. Application of this framework reveals a highly “jagged” cognitive profile in contemporary models. While proficient in knowledge-intensive domains, current AI systems have critical deficits in foundational cognitive machinery, particularly long-term memory storage. The resulting AGI scores (e.g., GPT-4 at 27%, GPT-5 at 57%) concretely quantify both rapid progress and the substantial gap remaining before AGI.

Method: Used embedding models to measure semantic similarities between extracted and original training data, rather than relying on approximate string matching which misses significant data due to trivial artifacts.

Result: Found that models readily regurgitate training data from post-training phases (SFT/RL), and this extracted data can train base models to recover meaningful performance. String matching would have underestimated extraction by 10x.

Conclusion: The work exposes overlooked risks in alignment data extraction and suggests that distillation practices may indirectly train models on original datasets through data regurgitation.

Abstract: In this work, we show that it is possible to extract significant amounts of alignment training data from a post-trained model – useful to steer the model to improve certain capabilities such as long-context reasoning, safety, instruction following, and maths. While the majority of related work on memorisation has focused on measuring success of training data extraction through string matching, we argue that embedding models are better suited for our specific goals. Distances measured through a high quality embedding model can identify semantic similarities between strings that a different metric such as edit distance will struggle to capture. In fact, in our investigation, approximate string matching would have severely undercounted (by a conservative estimate of $10\times$) the amount of data that can be extracted due to trivial artifacts that deflate the metric. Interestingly, we find that models readily regurgitate training data that was used in post-training phases such as SFT or RL. We show that this data can be then used to train a base model, recovering a meaningful amount of the original performance. We believe our work exposes a possibly overlooked risk towards extracting alignment data. Finally, our work opens up an interesting discussion on the downstream effects of distillation practices: since models seem to be regurgitating aspects of their training set, distillation can therefore be thought of as indirectly training on the model’s original dataset.

Method: Integrates Bayesian Experimental Design with large language models to select tests that maximize diagnostic uncertainty reduction. LLMs act as flexible simulators for patient state distributions and belief updates without requiring structured training data.

Result: ACTMED optimizes test selection to improve diagnostic accuracy, interpretability, and resource use on real-world datasets.

Conclusion: Represents progress toward transparent, adaptive diagnostic systems that generalize across settings with reduced domain-specific data requirements while keeping clinicians in the loop.

Abstract: There is growing interest in using machine learning (ML) to support clinical diagnosis, but most approaches rely on static, fully observed datasets and fail to reflect the sequential, resource-aware reasoning clinicians use in practice. Diagnosis remains complex and error prone, especially in high-pressure or resource-limited settings, underscoring the need for frameworks that help clinicians make timely and cost-effective decisions. We propose ACTMED (Adaptive Clinical Test selection via Model-based Experimental Design), a diagnostic framework that integrates Bayesian Experimental Design (BED) with large language models (LLMs) to better emulate real-world diagnostic reasoning. At each step, ACTMED selects the test expected to yield the greatest reduction in diagnostic uncertainty for a given patient. LLMs act as flexible simulators, generating plausible patient state distributions and supporting belief updates without requiring structured, task-specific training data. Clinicians can remain in the loop; reviewing test suggestions, interpreting intermediate outputs, and applying clinical judgment throughout. We evaluate ACTMED on real-world datasets and show it can optimize test selection to improve diagnostic accuracy, interpretability, and resource use. This represents a step toward transparent, adaptive, and clinician-aligned diagnostic systems that generalize across settings with reduced reliance on domain-specific data.

Method: Uses a training-free multi-agent evolution system with LLMs, starting with classical planner baseline solutions and iteratively refining them to adapt to dynamic urban conditions and worker preferences.

Result: Outperforms traditional methods in adaptivity and explainability, and beats single-agent LLM baselines in performance, robustness, and explanation quality across large-scale mobility datasets and dynamic disturbances.

Conclusion: AgentSense represents a significant advancement towards deploying adaptive and explainable urban sensing systems on the web.

Abstract: Web-based participatory urban sensing has emerged as a vital approach for modern urban management by leveraging mobile individuals as distributed sensors. However, existing urban sensing systems struggle with limited generalization across diverse urban scenarios and poor interpretability in decision-making. In this work, we introduce AgentSense, a hybrid, training-free framework that integrates large language models (LLMs) into participatory urban sensing through a multi-agent evolution system. AgentSense initially employs classical planner to generate baseline solutions and then iteratively refines them to adapt sensing task assignments to dynamic urban conditions and heterogeneous worker preferences, while producing natural language explanations that enhance transparency and trust. Extensive experiments across two large-scale mobility datasets and seven types of dynamic disturbances demonstrate that AgentSense offers distinct advantages in adaptivity and explainability over traditional methods. Furthermore, compared to single-agent LLM baselines, our approach outperforms in both performance and robustness, while delivering more reasonable and transparent explanations. These results position AgentSense as a significant advancement towards deploying adaptive and explainable urban sensing systems on the web.

Method: Surfer 2 integrates hierarchical context management, decoupled planning and execution, and self-verification with adaptive recovery to enable reliable operation over long task horizons.

Result: Achieves 97.1% accuracy on WebVoyager, 69.6% on WebArena, 60.1% on OSWorld, and 87.1% on AndroidWorld, outperforming all prior systems. With multiple attempts, exceeds human performance on all benchmarks.

Conclusion: Systematic orchestration amplifies foundation model capabilities and enables general-purpose computer control through visual interaction alone, while calling for next-generation vision language models for Pareto-optimal cost-efficiency.

Abstract: Building agents that generalize across web, desktop, and mobile environments remains an open challenge, as prior systems rely on environment-specific interfaces that limit cross-platform deployment. We introduce Surfer 2, a unified architecture operating purely from visual observations that achieves state-of-the-art performance across all three environments. Surfer 2 integrates hierarchical context management, decoupled planning and execution, and self-verification with adaptive recovery, enabling reliable operation over long task horizons. Our system achieves 97.1% accuracy on WebVoyager, 69.6% on WebArena, 60.1% on OSWorld, and 87.1% on AndroidWorld, outperforming all prior systems without task-specific fine-tuning. With multiple attempts, Surfer 2 exceeds human performance on all benchmarks. These results demonstrate that systematic orchestration amplifies foundation model capabilities and enables general-purpose computer control through visual interaction alone, while calling for a next-generation vision language model to achieve Pareto-optimal cost-efficiency.

Method: A simple and efficient protocol using modest 8-billion-parameter open-source LLMs to encode and decode hidden messages within coherent text of the same length.

Result: High-quality results achieved with local processing on a laptop in seconds; messages as long as abstracts can be successfully encoded and decoded.

Conclusion: This technique demonstrates radical decoupling of text from intent, raises urgent AI safety concerns, and challenges our understanding of what LLMs truly know.

Abstract: A meaningful text can be hidden inside another, completely different yet still coherent and plausible, text of the same length. For example, a tweet containing a harsh political critique could be embedded in a tweet that celebrates the same political leader, or an ordinary product review could conceal a secret manuscript. This uncanny state of affairs is now possible thanks to Large Language Models, and in this paper we present a simple and efficient protocol to achieve it. We show that even modest 8-billion-parameter open-source LLMs are sufficient to obtain high-quality results, and a message as long as this abstract can be encoded and decoded locally on a laptop in seconds. The existence of such a protocol demonstrates a radical decoupling of text from authorial intent, further eroding trust in written communication, already shaken by the rise of LLM chatbots. We illustrate this with a concrete scenario: a company could covertly deploy an unfiltered LLM by encoding its answers within the compliant responses of a safe model. This possibility raises urgent questions for AI safety and challenges our understanding of what it means for a Large Language Model to know something.

Method: Uses a clustering approach that treats tokens within the same cluster equivalently to address the retokenization mismatch problem in autoregressive audio models, creating a distortion-free watermark specifically designed for audio generation.

Result: Comprehensive testing shows Aligned-IS preserves generated audio quality while significantly improving watermark detectability compared to state-of-the-art distortion-free watermarking adaptations.

Conclusion: Aligned-IS establishes a new benchmark in secure audio technology applications by effectively solving the retokenization mismatch challenge in audio watermarking.

Abstract: The rapid advancement of next-token-prediction models has led to widespread adoption across modalities, enabling the creation of realistic synthetic media. In the audio domain, while autoregressive speech models have propelled conversational interactions forward, the potential for misuse, such as impersonation in phishing schemes or crafting misleading speech recordings, has also increased. Security measures such as watermarking have thus become essential to ensuring the authenticity of digital media. Traditional statistical watermarking methods used for autoregressive language models face challenges when applied to autoregressive audio models, due to the inevitable ``retokenization mismatch’’ - the discrepancy between original and retokenized discrete audio token sequences. To address this, we introduce Aligned-IS, a novel, distortion-free watermark, specifically crafted for audio generation models. This technique utilizes a clustering approach that treats tokens within the same cluster equivalently, effectively countering the retokenization mismatch issue. Our comprehensive testing on prevalent audio generation platforms demonstrates that Aligned-IS not only preserves the quality of generated audio but also significantly improves the watermark detectability compared to the state-of-the-art distortion-free watermarking adaptations, establishing a new benchmark in secure audio technology applications.

Method: Hybrid acoustic simulation using finite-difference time-domain for low frequencies (up to 900 Hz) and ray-tracing for high frequencies (above 900 Hz), applied to geometrically complex furnished rooms from 3D-FRONT repository, with encoding into spherical-harmonic domain (AmbiX ACN).

Result: Created a dataset of over 100,000 7th-order Ambisonic RIRs with detailed statistics including room volumes, RT60 distributions, and absorption properties, offering improved accuracy compared to existing RIR collections.

Conclusion: HiFi-HARP provides a comprehensive resource for spatial audio research with potential applications in FOA-to-HOA upsampling, source localization, dereverberation, and machine learning tasks, though limited by simulation approximations and static scenes.

Abstract: We introduce HiFi-HARP, a large-scale dataset of 7th-order Higher-Order Ambisonic Room Impulse Responses (HOA-RIRs) consisting of more than 100,000 RIRs generated via a hybrid acoustic simulation in realistic indoor scenes. HiFi-HARP combines geometrically complex, furnished room models from the 3D-FRONT repository with a hybrid simulation pipeline: low-frequency wave-based simulation (finite-difference time-domain) up to 900 Hz is used, while high frequencies above 900 Hz are simulated using a ray-tracing approach. The combined raw RIRs are encoded into the spherical-harmonic domain (AmbiX ACN) for direct auralization. Our dataset extends prior work by providing 7th-order Ambisonic RIRs that combine wave-theoretic accuracy with realistic room content. We detail the generation pipeline (scene and material selection, array design, hybrid simulation, ambisonic encoding) and provide dataset statistics (room volumes, RT60 distributions, absorption properties). A comparison table highlights the novelty of HiFi-HARP relative to existing RIR collections. Finally, we outline potential benchmarks such as FOA-to-HOA upsampling, source localization, and dereverberation. We discuss machine learning use cases (spatial audio rendering, acoustic parameter estimation) and limitations (e.g., simulation approximations, static scenes). Overall, HiFi-HARP offers a rich resource for developing spatial audio and acoustics algorithms in complex environments.

Method: Uses conditional separation with prompt vectors (one per speaker) and an array-agnostic channel communication mechanism to process multi-channel mixtures with arbitrary microphone configurations.

Result: Successfully handles diverse conditions from 1-5 microphones and 1-3 speakers. Demonstrates robustness on CHiME-4 real data.

Conclusion: FlexIO provides a unified solution for flexible speech separation that accommodates both variable input configurations and variable output speakers, overcoming limitations of previous specialized systems.

Abstract: Speech separation and enhancement (SSE) has advanced remarkably and achieved promising results in controlled settings, such as a fixed number of speakers and a fixed array configuration. Towards a universal SSE system, single-channel systems have been extended to deal with a variable number of speakers (i.e., outputs). Meanwhile, multi-channel systems accommodating various array configurations (i.e., inputs) have been developed. However, these attempts have been pursued separately. In this paper, we propose a flexible input and output SSE system, named FlexIO. It performs conditional separation using prompt vectors, one per speaker as a condition, allowing separation of an arbitrary number of speakers. Multi-channel mixtures are processed together with the prompt vectors via an array-agnostic channel communication mechanism. Our experiments demonstrate that FlexIO successfully covers diverse conditions with one to five microphones and one to three speakers. We also confirm the robustness of FlexIO on CHiME-4 real data.

Method: Single-stage model performing end-to-end vocal restoration in complex STFT domain with phase-aware losses, enabling large analysis windows for improved frequency resolution.

Result: Outperforms strong GAN baseline on DNS 5 Challenge, matches computationally expensive flow-matching system, and surpasses open-source baselines on singing while matching commercial systems on Extreme Degradation Bench.

Conclusion: SRS provides efficient vocal restoration for multiple degradations with mobile-friendly inference, and the released EDB benchmark enables realistic multi-degradation evaluation.

Abstract: Vocal recordings on consumer devices commonly suffer from multiple concurrent degradations: noise, reverberation, band-limiting, and clipping. We present Smule Renaissance Small (SRS), a compact single-stage model that performs end-to-end vocal restoration directly in the complex STFT domain. By incorporating phase-aware losses, SRS enables large analysis windows for improved frequency resolution while achieving 10.5x real-time inference on iPhone 12 CPU at 48 kHz. On the DNS 5 Challenge blind set, despite no speech training, SRS outperforms a strong GAN baseline and closely matches a computationally expensive flow-matching system. To enable evaluation under realistic multi-degradation scenarios, we introduce the Extreme Degradation Bench (EDB): 87 singing and speech recordings captured under severe acoustic conditions. On EDB, SRS surpasses all open-source baselines on singing and matches commercial systems, while remaining competitive on speech despite no speech-specific training. We release both SRS and EDB under the MIT License.

Method: Uses distributional flow matching (DFM) that parameterizes velocity field as Gaussian distribution with negative log-likelihood optimization, enabling uncertainty modeling. Combines with test-time optimization that samples multiple trajectories weighted by model confidence and selects outputs maximizing timbre consistency.

Result: Outperforms state-of-the-art TokenSynth baseline in both single-note quality and cross-note consistency.

Conclusion: Modeling predictive uncertainty in flow matching combined with music-specific consistency objectives provides an effective path to professional-quality virtual instruments suitable for real-time performance.

Abstract: Virtual instrument generation requires maintaining consistent timbre across different pitches and velocities, a challenge that existing note-level models struggle to address. We present FlowSynth, which combines distributional flow matching (DFM) with test-time optimization for high-quality instrument synthesis. Unlike standard flow matching that learns deterministic mappings, DFM parameterizes the velocity field as a Gaussian distribution and optimizes via negative log-likelihood, enabling the model to express uncertainty in its predictions. This probabilistic formulation allows principled test-time search: we sample multiple trajectories weighted by model confidence and select outputs that maximize timbre consistency. FlowSynth outperforms the current state-of-the-art TokenSynth baseline in both single-note quality and cross-note consistency. Our approach demonstrates that modeling predictive uncertainty in flow matching, combined with music-specific consistency objectives, provides an effective path to professional-quality virtual instruments suitable for real-time performance.

Method: Built on rectified flow matching architecture, StylePitcher incorporates symbolic music scores and pitch context as conditions, learning singer style from reference audio while preserving melody alignment.

Result: Objective and subjective evaluations across various singing tasks show improved style similarity and audio quality while maintaining pitch accuracy comparable to task-specific baselines.

Conclusion: StylePitcher provides a flexible, general-purpose solution for pitch curve generation that can seamlessly adapt to diverse singing tasks while capturing individual singing styles.

Abstract: Existing pitch curve generators face two main challenges: they often neglect singer-specific expressiveness, reducing their ability to capture individual singing styles. And they are typically developed as auxiliary modules for specific tasks such as pitch correction, singing voice synthesis, or voice conversion, which restricts their generalization capability. We propose StylePitcher, a general-purpose pitch curve generator that learns singer style from reference audio while preserving alignment with the intended melody. Built upon a rectified flow matching architecture, StylePitcher flexibly incorporates symbolic music scores and pitch context as conditions for generation, and can seamlessly adapt to diverse singing tasks without retraining. Objective and subjective evaluations across various singing tasks demonstrate that StylePitcher improves style similarity and audio quality while maintaining pitch accuracy comparable to task-specific baselines.

Method: Systematic assessment of leading multimodal models benchmarked against human performance across six audiovisual conditions. Proposed EchoPin model using stereo audio-image dataset generated via 3D simulations.

Result: Humans consistently outperform AI, showing superior resilience to conflicting/missing visuals. AI models degrade to near chance levels by favoring vision. EchoPin surpasses existing benchmarks even with limited training data and exhibits human-like horizontal localization bias.

Conclusion: Sensory input quality and system architecture significantly shape multimodal representation accuracy, with stereo audio structure enabling more human-like performance in sound localization.

Abstract: Imagine hearing a dog bark and turning toward the sound only to see a parked car, while the real, silent dog sits elsewhere. Such sensory conflicts test perception, yet humans reliably resolve them by prioritizing sound over misleading visuals. Despite advances in multimodal AI integrating vision and audio, little is known about how these systems handle cross-modal conflicts or whether they favor one modality. In this study, we systematically examine modality bias and conflict resolution in AI sound localization. We assess leading multimodal models and benchmark them against human performance in psychophysics experiments across six audiovisual conditions, including congruent, conflicting, and absent cues. Humans consistently outperform AI, demonstrating superior resilience to conflicting or missing visuals by relying on auditory information. In contrast, AI models often default to visual input, degrading performance to near chance levels. To address this, we propose a neuroscience-inspired model, EchoPin, which uses a stereo audio-image dataset generated via 3D simulations. Even with limited training data, EchoPin surpasses existing benchmarks. Notably, it also mirrors human-like horizontal localization bias favoring left-right precision-likely due to the stereo audio structure reflecting human ear placement. These findings underscore how sensory input quality and system architecture shape multimodal representation accuracy.

Method: Two-step approach: 1) Feature extraction using VAEs on waveforms and spectrograms to avoid manual thresholding, 2) Classification using Temporal Convolutional Networks trained on either VAE embeddings or handpicked features.

Result: TCNs achieved over 85% accuracy on 4-minute recordings. VAE-based features showed more consistent performance across diverse deployment conditions compared to handpicked features.

Conclusion: VAE-based feature extraction provides robust transferability across different deployment conditions, making it superior to traditional handpicked features for weakly labeled PAM data.

Abstract: Passive acoustic monitoring (PAM) data is often weakly labelled, audited at the scale of detection presence or absence on timescales of minutes to hours. Moreover, this data exhibits great variability from one deployment to the next, due to differences in ambient noise and the signals across sources and geographies. This study proposes a two-step solution to leverage weakly annotated data for training Deep Learning (DL) detection models. Our case study involves binary classification of the presence/absence of sperm whale (\textit{Physeter macrocephalus}) click trains in 4-minute-long recordings from a dataset comprising diverse sources and deployment conditions to maximise generalisability. We tested methods for extracting acoustic features from lengthy audio segments and integrated Temporal Convolutional Networks (TCNs) trained on the extracted features for sequence classification. For feature extraction, we introduced a new approach using Variational AutoEncoders (VAEs) to extract information from both waveforms and spectrograms, which eliminates the necessity for manual threshold setting or time-consuming strong labelling. For classification, TCNs were trained separately on sequences of either VAE embeddings or handpicked acoustic features extracted from the waveform and spectrogram representations using classical methods, to compare the efficacy of the two approaches. The TCN demonstrated robust classification capabilities on a validation set, achieving accuracies exceeding 85% when applied to 4-minute acoustic recordings. Notably, TCNs trained on handpicked acoustic features exhibited greater variability in performance across recordings from diverse deployment conditions, whereas those trained on VAEs showed a more consistent performance, highlighting the robust transferability of VAEs for feature extraction across different deployment conditions.

Method: Analyzed PTTM performance in various noise types and signal-to-noise ratios (SNRs), trained multimodal PTTMs with noisy data that incorporate visual features, and compared audio-only vs multimodal approaches.

Result: PTTMs show significant performance degradation in noise (84% to 52% accuracy). Multimodal PTTMs trained with noisy data achieve 72% accuracy in 10 dB music noise and outperform audio-only models across all noise types and SNRs, though generalization to new noise types is limited.

Conclusion: Multimodal PTTMs trained with noisy data can better handle noise by exploiting visual cues, but successful training requires accurate transcription and performance doesn’t always generalize to new noise types.

Abstract: Accurate predictive turn-taking models (PTTMs) are essential for naturalistic human-robot interaction. However, little is known about their performance in noise. This study therefore explores PTTM performance in types of noise likely to be encountered once deployed. Our analyses reveal PTTMs are highly sensitive to noise. Hold/shift accuracy drops from 84% in clean speech to just 52% in 10 dB music noise. Training with noisy data enables a multimodal PTTM, which includes visual features to better exploit visual cues, with 72% accuracy in 10 dB music noise. The multimodal PTTM outperforms the audio-only PTTM across all noise types and SNRs, highlighting its ability to exploit visual cues; however, this does not always generalise to new types of noise. Analysis also reveals that successful training relies on accurate transcription, limiting the use of ASR-derived transcriptions to clean conditions. We make code publicly available for future research.

Method: Developed SHAP-based interpretability analysis to quantify speaker information in embeddings, then used these insights to filter speaker information without retraining models.

Result: SHAP Noise filtering reduced speaker residuals from 18.05% to nearly zero while maintaining recognition accuracy (CTC loss increase under 1%) on VCTK dataset with seven models including HuBERT, WavLM, and ContentVec.

Conclusion: The proposed method effectively disentangles speaker and content information in speech embeddings, is model-agnostic, and requires no retraining, addressing both speaker bias and privacy concerns.

Abstract: Self-supervised speech models learn representations that capture both content and speaker information. Yet this entanglement creates problems: content tasks suffer from speaker bias, and privacy concerns arise when speaker identity leaks through supposedly anonymized representations. We present two contributions to address these challenges. First, we develop InterpTRQE-SptME (Timbre Residual Quantitative Evaluation Benchmark of Speech pre-training Models Encoding via Interpretability), a benchmark that directly measures residual speaker information in content embeddings using SHAP-based interpretability analysis. Unlike existing indirect metrics, our approach quantifies the exact proportion of speaker information remaining after disentanglement. Second, we propose InterpTF-SptME, which uses these interpretability insights to filter speaker information from embeddings. Testing on VCTK with seven models including HuBERT, WavLM, and ContentVec, we find that SHAP Noise filtering reduces speaker residuals from 18.05% to nearly zero while maintaining recognition accuracy (CTC loss increase under 1%). The method is model-agnostic and requires no retraining.

Method: Online reinforcement learning with Group Relative Policy Optimization and multi-faceted rewards that incentivize correctness, format, consistency, structured analysis, causal reasoning, domain knowledge, and calibrated reasoning depth.

Result: Achieved state-of-the-art on MMAU Test-mini, outperforming Gemini 2.5 Pro and GPT-4o Audio, near-human performance on MMSU reasoning tasks, and resolved test-time inverse scaling.

Conclusion: CESAR establishes a principled method for developing robust and scalable reasoning in Audio LLMs, transforming reasoning from detrimental to beneficial while creating synergistic improvements in multimodal capabilities.

Abstract: The role of reasoning in Audio Large Language Models remains widely underexplored, as introducing a reasoning process often degrades rather than improves performance during inference, a phenomenon we term test-time inverse scaling, where longer reasoning chains yield progressively worse results. We demonstrate that this stems not from fundamental limitations of reasoning itself, but from inadequate training: models without proper guidance for the reasoning process produce hallucinatory, inconsistent reasoning that accumulates errors over longer chains. To address these challenges, we introduce CESAR (Consistent, Effective, and Scalable Audio Reasoners), shifting from outcome verification to rewarding the reasoning process. Our online reinforcement learning framework employs Group Relative Policy Optimization with a multi-faceted reward suite that incentivizes not only correctness and format but also consistency, structured analytical patterns, causal reasoning, domain-knowledge integration, and calibrated reasoning depth. CESAR resolves test-time inverse scaling, transforming reasoning from detriments into gains while revealing model-specific ``reasoning sweet spots", where performance peaks during test-time scaling. We achieve state-of-the-art results on MMAU Test-mini, substantially outperforming Gemini 2.5 Pro and GPT-4o Audio, and near-human-level performance on MMSU reasoning tasks. Through AI-as-judge evaluations and qualitative comparisons, we provide both quantitative and qualitative validation of our improved reasoning quality. Importantly, enhanced reasoning creates synergistic effects, simultaneously improving multimodal reasoning and perception capabilities. Overall, CESAR establishes a principled method for developing robust and scalable reasoning in Audio LLMs.

Method: Uses Graph Convolutional Networks for spatial relationships, BiLSTM with self-attention for temporal dynamics, and multi-head Graph Attention Networks to learn sparse structures and discover which asset relationships matter through attention mechanisms.

Result: Achieves Sharpe ratio 3.76 (707% improvement over equal-weight baselines, 94% improvement over static graph methods), filters 92.5% of connections as noise while preserving crisis-relevant dependencies, and generalizes well to fundamentally different market regimes.

Conclusion: CRISP enables adaptive portfolio allocation that maintains profitability during downturns, with learned attention weights providing interpretable regime detection and emergent behavior from learning rather than imposing assumptions.

Abstract: Financial time series forecasting faces a fundamental challenge: predicting optimal asset allocations requires understanding regime-dependent correlation structures that transform during crisis periods. Existing graph-based spatio-temporal learning approaches rely on predetermined graph topologies–correlation thresholds, sector classifications–that fail to adapt when market dynamics shift across different crisis mechanisms: credit contagion, pandemic shocks, or inflation-driven selloffs. We present CRISP (Crisis-Resilient Investment through Spatio-temporal Patterns), a graph-based spatio-temporal learning framework that encodes spatial relationships via Graph Convolutional Networks and temporal dynamics via BiLSTM with self-attention, then learns sparse structures through multi-head Graph Attention Networks. Unlike fixed-topology methods, CRISP discovers which asset relationships matter through attention mechanisms, filtering 92.5% of connections as noise while preserving crisis-relevant dependencies for accurate regime-specific predictions. Trained on 2005–2021 data encompassing credit and pandemic crises, CRISP demonstrates robust generalization to 2022–2024 inflation-driven markets–a fundamentally different regime–by accurately forecasting regime-appropriate correlation structures. This enables adaptive portfolio allocation that maintains profitability during downturns, achieving Sharpe ratio 3.76: 707% improvement over equal-weight baselines and 94% improvement over static graph methods. Learned attention weights provide interpretable regime detection, with defensive cluster attention strengthening 49% during crises versus 31% market-wide–emergent behavior from learning to forecast rather than imposing assumptions.

Method: Proposes MOBO-OSD which solves multiple constrained optimization problems along orthogonal search directions defined with respect to an approximated convex hull of individual objective minima, uses Pareto Front Estimation for solution density, and supports batch optimization for parallel evaluations.

Result: Extensive experiments on synthetic and real-world benchmarks with 2-6 objectives show MOBO-OSD consistently outperforms state-of-the-art algorithms in solution diversity and hypervolume performance.

Conclusion: MOBO-OSD effectively addresses multi-objective optimization challenges by ensuring broad objective space coverage and solution density through orthogonal search directions and Pareto Front Estimation, with demonstrated superior performance across various benchmarks.

Abstract: Bayesian Optimization (BO) is a powerful tool for optimizing expensive black-box objective functions. While extensive research has been conducted on the single-objective optimization problem, the multi-objective optimization problem remains challenging. In this paper, we propose MOBO-OSD, a multi-objective Bayesian Optimization algorithm designed to generate a diverse set of Pareto optimal solutions by solving multiple constrained optimization problems, referred to as MOBO-OSD subproblems, along orthogonal search directions (OSDs) defined with respect to an approximated convex hull of individual objective minima. By employing a well-distributed set of OSDs, MOBO-OSD ensures broad coverage of the objective space, enhancing both solution diversity and hypervolume performance. To further improve the density of the set of Pareto optimal candidate solutions without requiring an excessive number of subproblems, we leverage a Pareto Front Estimation technique to generate additional solutions in the neighborhood of existing solutions. Additionally, MOBO-OSD supports batch optimization, enabling parallel function evaluations to accelerate the optimization process when resources are available. Through extensive experiments and analysis on a variety of synthetic and real-world benchmark functions with two to six objectives, we demonstrate that MOBO-OSD consistently outperforms the state-of-the-art algorithms. Our code implementation can be found at https://github.com/LamNgo1/mobo-osd.

Method: A three-agent system (Prediction, Planning, Execution) that leverages satellite observations and environmental signals, incorporating local factors and multi-agent coordination for real-time situational awareness and intervention.

Result: The framework enhances landslide forecasting accuracy and enables real-time response planning and intervention through collaborative multi-agent coordination.

Conclusion: CC-GRMAS offers a scalable, proactive solution for climate-resilient disaster preparedness in vulnerable mountainous regions by operationalizing multi-agent coordination with environmental data.

Abstract: Landslides are a growing climate induced hazard with severe environmental and human consequences, particularly in high mountain Asia. Despite increasing access to satellite and temporal datasets, timely detection and disaster response remain underdeveloped and fragmented. This work introduces CC-GRMAS, a framework leveraging a series of satellite observations and environmental signals to enhance the accuracy of landslide forecasting. The system is structured around three interlinked agents Prediction, Planning, and Execution, which collaboratively enable real time situational awareness, response planning, and intervention. By incorporating local environmental factors and operationalizing multi agent coordination, this approach offers a scalable and proactive solution for climate resilient disaster preparedness across vulnerable mountainous terrains.

Method: Introduces Negative Learning paradigm where dominant modalities dynamically guide weak modality to suppress non-target classes instead of enhancing target-class predictions. Uses dynamic guidance mechanism to stabilize decision space and preserve modality-specific information.

Result: Theoretically tightens robustness lower bound by increasing Unimodal Confidence Margin (UCoM) and reduces empirical error of weak modalities, especially under noisy and imbalanced scenarios. Extensive experiments show effectiveness and generalizability across multiple benchmarks.

Conclusion: MNL framework successfully addresses modality imbalance by preserving unique information in weak modalities through negative learning, outperforming competing methods while providing theoretical robustness guarantees.

Abstract: Multimodal learning systems often encounter challenges related to modality imbalance, where a dominant modality may overshadow others, thereby hindering the learning of weak modalities. Conventional approaches often force weak modalities to align with dominant ones in “Learning to be (the same)” (Positive Learning), which risks suppressing the unique information inherent in the weak modalities. To address this challenge, we offer a new learning paradigm: “Learning Not to be” (Negative Learning). Instead of enhancing weak modalities’ target-class predictions, the dominant modalities dynamically guide the weak modality to suppress non-target classes. This stabilizes the decision space and preserves modality-specific information, allowing weak modalities to preserve unique information without being over-aligned. We proceed to reveal multimodal learning from a robustness perspective and theoretically derive the Multimodal Negative Learning (MNL) framework, which introduces a dynamic guidance mechanism tailored for negative learning. Our method provably tightens the robustness lower bound of multimodal learning by increasing the Unimodal Confidence Margin (UCoM) and reduces the empirical error of weak modalities, particularly under noisy and imbalanced scenarios. Extensive experiments across multiple benchmarks demonstrate the effectiveness and generalizability of our approach against competing methods. The code will be available at https://github.com/BaoquanGong/Multimodal-Negative-Learning.git.

Method: 1) Hotness-aware mixed-precision compression and loading based on KV chunk access frequency; 2) Hotness-aware data placement strategy that prioritizes frequently accessed KV chunks in high-speed memory.

Result: Achieved average speedup of 2.10x and maximum speedup of 10.49x in Time-To-First-Token compared to TurboRAG, with negligible accuracy loss.

Conclusion: HA-RAG effectively optimizes RAG inference by leveraging access patterns to reduce I/O overhead and improve data access efficiency, making it a practical solution for production RAG systems.

Abstract: Retrieval-Augmented Generation (RAG) improves model output accuracy by leveraging external knowledge bases, serving as an effective solution to address hallucination issues and knowledge-update delays in Large Language Models (LLMs). However, the introduction of external knowledge bases presents RAG with challenges in long-context processing, significantly increasing memory consumption and inference latency. Existing research accelerates inference by precomputing Key and Value (KV) of the knowledge base and loading them on-demand during inference. Based on the access frequency of different KV chunks within the external knowledge base, this paper proposes a hotness-aware RAG (HA-RAG) inference optimization system. First, leveraging the numerical distribution of KV chunks, we introduce a hotness-aware mixed-precision compressing and loading method to reduce disk I/O and memory access overhead. Second, we design a hotness-aware data placement strategy that prioritizes storing frequently accessed KV chunks in high-speed memory to improve data access efficiency. Experimental results demonstrate that, compared with TurboRAG, the proposed HA-RAG achieves an average speedup of 2.10x and maximum speedup of 10.49x in Time-To-First-Token (TTFT) with negligible accuracy loss.

Method: Developed technical machinery based on large deviations and metastability analysis, injecting and truncating heavy-tailed noises during training with gradient clipping.

Result: Heavy-tailed SGD almost completely avoids sharp minima and finds local minima with flatter geometry, achieving better generalization performance.

Conclusion: Heavy-tailed SGD with gradient clipping is an effective approach for improving generalization by avoiding sharp local minima in deep learning.

Abstract: Stochastic gradient descent (SGD) and its variants enable modern artificial intelligence. However, theoretical understanding lags far behind their empirical success. It is widely believed that SGD has a curious ability to avoid sharp local minima in the loss landscape, which are associated with poor generalization. To unravel this mystery and further enhance such capability of SGDs, it is imperative to go beyond the traditional local convergence analysis and obtain a comprehensive understanding of SGDs’ global dynamics. In this paper, we develop a set of technical machinery based on the recent large deviations and metastability analysis in Wang and Rhee (2023) and obtain sharp characterization of the global dynamics of heavy-tailed SGDs. In particular, we reveal a fascinating phenomenon in deep learning: by injecting and then truncating heavy-tailed noises during the training phase, SGD can almost completely avoid sharp minima and achieve better generalization performance for the test data. Simulation and deep learning experiments confirm our theoretical prediction that heavy-tailed SGD with gradient clipping finds local minima with a more flat geometry and achieves better generalization performance.

Method: Two approaches: 1) Using loss functions compatible with interval targets with generalization bounds based on smoothness, 2) A min-max learning formulation that minimizes against worst-case target labels within intervals, incorporating smoothness constraints to handle non-convexity.

Result: Extensive experiments on real-world datasets demonstrate that the proposed methods achieve state-of-the-art performance.

Conclusion: The paper successfully addresses regression with interval targets through novel loss functions and min-max formulations with smoothness constraints, providing effective solutions for scenarios where exact targets are unavailable.

Abstract: We study the problem of regression with interval targets, where only upper and lower bounds on target values are available in the form of intervals. This problem arises when the exact target label is expensive or impossible to obtain, due to inherent uncertainties. In the absence of exact targets, traditional regression loss functions cannot be used. First, we study the methodology of using a loss functions compatible with interval targets, for which we establish non-asymptotic generalization bounds based on smoothness of the hypothesis class that significantly relaxing prior assumptions of realizability and small ambiguity degree. Second, we propose a novel min-max learning formulation: minimize against the worst-case (maximized) target labels within the provided intervals. The maximization problem in the latter is non-convex, but we show that good performance can be achieved with the incorporation of smoothness constraints. Finally, we perform extensive experiments on real-world datasets and show that our methods achieve state-of-the-art performance.

Method: Combines Model-Agnostic Meta-Learning (MAML) with transformer architectures and introduces a novel mutation encoding strategy using separator tokens to directly incorporate mutations into sequence context, enabling rapid adaptation to new tasks.

Result: Achieves 29% better accuracy for functional fitness with 65% less training time, and 94% better accuracy for solubility with 55% faster training across three diverse protein mutation datasets.

Conclusion: Establishes a systematic meta-learning framework for protein mutation analysis with effective mutation encoding, offering transformative methodology for cross-domain generalization in protein engineering applications.

Abstract: Protein mutations can have profound effects on biological function, making accurate prediction of property changes critical for drug discovery, protein engineering, and precision medicine. Current approaches rely on fine-tuning protein-specific transformers for individual datasets, but struggle with cross-dataset generalization due to heterogeneous experimental conditions and limited target domain data. We introduce two key innovations: (1) the first application of Model-Agnostic Meta-Learning (MAML) to protein mutation property prediction, and (2) a novel mutation encoding strategy using separator tokens to directly incorporate mutations into sequence context. We build upon transformer architectures integrating them with MAML to enable rapid adaptation to new tasks through minimal gradient steps rather than learning dataset-specific patterns. Our mutation encoding addresses the critical limitation where standard transformers treat mutation positions as unknown tokens, significantly degrading performance. Evaluation across three diverse protein mutation datasets (functional fitness, thermal stability, and solubility) demonstrates significant advantages over traditional fine-tuning. In cross-task evaluation, our meta-learning approach achieves 29% better accuracy for functional fitness with 65% less training time, and 94% better accuracy for solubility with 55% faster training. The framework maintains consistent training efficiency regardless of dataset size, making it particularly valuable for industrial applications and early-stage protein design where experimental data is limited. This work establishes a systematic application of meta-learning to protein mutation analysis and introduces an effective mutation encoding strategy, offering transformative methodology for cross-domain generalization in protein engineering.

Method: Combines state space models for temporal dynamics with adapted LLMs for encoding textual inputs and decoding textual forecasts consistent with latent states.

Result: Achieves 13.20% improvement over previous state-of-the-art on TextTimeCorpus benchmark while providing human-readable forecast summaries.

Conclusion: First framework to unify LLMs and SSMs for joint numerical and textual prediction, offering a novel foundation for multimodal temporal reasoning.

Abstract: Forecasting in the real world requires integrating structured time-series data with unstructured textual information, but existing methods are architecturally limited by fixed input/output horizons and are unable to model or quantify uncertainty. We address this challenge by introducing LLM-integrated Bayesian State space models (LBS), a novel probabilistic framework for multimodal temporal forecasting. At a high level, LBS consists of two components: (1) a state space model (SSM) backbone that captures the temporal dynamics of latent states from which both numerical and textual observations are generated and (2) a pretrained large language model (LLM) that is adapted to encode textual inputs for posterior state estimation and decode textual forecasts consistent with the latent trajectory. This design enables flexible lookback and forecast windows, principled uncertainty quantification, and improved temporal generalization thanks to the well-suited inductive bias of SSMs toward modeling dynamical systems. Experiments on the TextTimeCorpus benchmark demonstrate that LBS improves the previous state-of-the-art by 13.20% while providing human-readable summaries of each forecast. Our work is the first to unify LLMs and SSMs for joint numerical and textual prediction, offering a novel foundation for multimodal temporal reasoning.

Method: SUBSAMPLE-MFQ algorithm with decentralized randomized policy that subsamples k agents and learns policies in time polynomial in k.

Result: Learned policy converges to optimal policy at rate O~(1/√k), independent of total number of agents n.

Conclusion: The approach provides scalable MARL solution with convergence guarantees that don’t depend on system size, addressing fundamental scalability challenges.

Abstract: Designing efficient algorithms for multi-agent reinforcement learning (MARL) is fundamentally challenging because the size of the joint state and action spaces grows exponentially in the number of agents. These difficulties are exacerbated when balancing sequential global decision-making with local agent interactions. In this work, we propose a new algorithm $\texttt{SUBSAMPLE-MFQ}$ ($\textbf{Subsample}$-$\textbf{M}$ean-$\textbf{F}$ield-$\textbf{Q}$-learning) and a decentralized randomized policy for a system with $n$ agents. For any $k\leq n$, our algorithm learns a policy for the system in time polynomial in $k$. We prove that this learned policy converges to the optimal policy on the order of $\tilde{O}(1/\sqrt{k})$ as the number of subsampled agents $k$ increases. In particular, this bound is independent of the number of agents $n$.

Method: Leverages reversibility to prevent safety issues using model-predictive path integral control to check safety of actions from learned policies, requiring only black-box dynamics queries.

Result: Successfully aborts before all unsafe actions while achieving comparable training progress to baseline PPO that violates safety.

Conclusion: Reversibility-based safety checking using model-predictive control provides effective safety guarantees without requiring explicit dynamics or safety constraint knowledge.

Abstract: Model-free reinforcement learning approaches are promising for control but typically lack formal safety guarantees. Existing methods to shield or otherwise provide these guarantees often rely on detailed knowledge of the safety specifications. Instead, this work’s insight is that many difficult-to-specify safety issues are best characterized by invariance. Accordingly, we propose to leverage reversibility as a method for preventing these safety issues throughout the training process. Our method uses model-predictive path integral control to check the safety of an action proposed by a learned policy throughout training. A key advantage of this approach is that it only requires the ability to query the black-box dynamics, not explicit knowledge of the dynamics or safety constraints. Experimental results demonstrate that the proposed algorithm successfully aborts before all unsafe actions, while still achieving comparable training progress to a baseline PPO approach that is allowed to violate safety.

Method: EPFL integrates continual learning, personalized modeling, and Specialized Weighted Aggregation (SWA) strategy using wearable sensor data. It employs penalized local training, ensemble-based inference, homomorphic encryption, and federated training to preserve privacy while improving adaptability.

Result: Extensive experiments show EPFL achieves 88.31% Recall and 89.94% F1-score, significantly outperforming both centralized and baseline models.

Conclusion: EPFL presents a scalable, secure, and accurate solution for real-world fall detection in healthcare settings with strong potential for continuous improvement through its adaptive feedback mechanism.

Abstract: Falls among elderly and disabled individuals remain a leading cause of injury and mortality worldwide, necessitating robust, accurate, and privacy-aware fall detection systems. Traditional fall detection approaches, whether centralized or point-wise, often struggle with key challenges such as limited generalizability, data privacy concerns, and variability in individual movement behaviors. To address these limitations, we propose EPFL-an Ensembled Penalized Federated Learning framework that integrates continual learning, personalized modeling, and a novel Specialized Weighted Aggregation (SWA) strategy. EPFL leverages wearable sensor data to capture sequential motion patterns while preserving user privacy through homomorphic encryption and federated training. Unlike existing federated models, EPFL incorporates both penalized local training and ensemble-based inference to improve inter-client consistency and adaptability to behavioral differences. Extensive experiments on a benchmark fall detection dataset demonstrate the effectiveness of our approach, achieving a Recall of 88.31 percent and an F1-score of 89.94 percent, significantly outperforming both centralized and baseline models. This work presents a scalable, secure, and accurate solution for real-world fall detection in healthcare settings, with strong potential for continuous improvement via its adaptive feedback mechanism.

Method: ColMAD reframes MAD as a non-zero sum game where multiple agents collaboratively criticize each other in a supportive manner, allowing them to complement each other’s missing points and provide more comprehensive evidence to the judge.

Result: ColMAD significantly outperforms previous competitive MAD by 19% and brings non-trivial improvements over single-agent methods in error detection tasks.

Conclusion: Collaborative multi-agent debate protocols that encourage supportive criticism can effectively mitigate debate hacking and improve error detection in large language models compared to competitive approaches.

Abstract: Accurate detection of errors in large language models (LLM) responses is central to the success of scalable oversight, or providing effective supervision to superhuman intelligence. Yet, self-diagnosis is often unreliable on complex tasks unless aided by reliable external feedback. Multi-agent debate (MAD) seems to be a natural alternative to external feedback: multiple LLMs provide complementary perspectives and cross-checks for error detection. However, prior MAD protocols frame debate as a zero-sum game, where the debaters compete to win the game instead of seeking the truth. Consequently, it leads to debate hacking: debaters tend to mislead the judge by misinterpreting the task or presenting overconfident claims, which introduce more mistakes and underperform single-agent methods. To mitigate the issue, we introduce a new collaborative MAD protocol, termed ColMAD, that reframes MAD as a non-zero sum game. Specifically, ColMAD encourages multiple agents to criticize each other in a supportive way, such that they can complement the missing points of each other. Therefore, the judge agent can make a more informative conclusion based on more comprehensive evidence. Empirically, we show that ColMAD significantly outperforms previous competitive MAD by 19% and brings non-trivial improvements over single-agent methods in error detection.

Method: Uses vector copula theory to create a principled interpolation between flexible neural network models and simplified Gaussian copula models.

Result: Demonstrates advantages over existing estimators on state-of-the-art synthetic benchmarks and real-world data with diverse modalities.

Conclusion: The proposed estimator achieves a better balance between complexity and capacity for mutual information estimation.

Abstract: Estimating mutual information (MI) is a fundamental task in data science and machine learning. Existing estimators mainly rely on either highly flexible models (e.g., neural networks), which require large amounts of data, or overly simplified models (e.g., Gaussian copula), which fail to capture complex distributions. Drawing upon recent vector copula theory, we propose a principled interpolation between these two extremes to achieve a better trade-off between complexity and capacity. Experiments on state-of-the-art synthetic benchmarks and real-world data with diverse modalities demonstrate the advantages of the proposed estimator.

Method: Investigated several strategies to mitigate spectral bias, including specialized activation functions, fixed and trainable positional encoding, and linear combinations of nonlinear kernels. Evaluated on hemodynamic fields from numerical simulations and cardiovascular anatomies via signed distance functions.

Result: On hemodynamic fields in the thoracic aorta, INRs achieved compression ratios up to ~230 with maximum absolute errors of 1 mmHg for pressure and 5-10 cm/s for velocity. Across 48 thoracic aortic anatomies, average and maximum absolute anatomical discrepancies were below 0.5 mm and 1.6 mm respectively. SIREN, MFN-Gabor, and MHE architectures performed best.

Conclusion: INRs offer resolution independence and high memory efficiency for representing hemodynamic fields and cardiovascular anatomies, achieving remarkable compression ratios with minimal errors without extensive hyperparameter tuning.

Abstract: Implicit neural representations (INRs, also known as neural fields) have recently emerged as a powerful framework for knowledge representation, synthesis, and compression. By encoding fields as continuous functions within the weights and biases of deep neural networks-rather than relying on voxel- or mesh-based structured or unstructured representations-INRs offer both resolution independence and high memory efficiency. However, their accuracy in domain-specific applications remains insufficiently understood. In this work, we assess the performance of state-of-the-art INRs for compressing hemodynamic fields derived from numerical simulations and for representing cardiovascular anatomies via signed distance functions. We investigate several strategies to mitigate spectral bias, including specialized activation functions, both fixed and trainable positional encoding, and linear combinations of nonlinear kernels. On realistic, space- and time-varying hemodynamic fields in the thoracic aorta, INRs achieved remarkable compression ratios of up to approximately 230, with maximum absolute errors of 1 mmHg for pressure and 5-10 cm/s for velocity, without extensive hyperparameter tuning. Across 48 thoracic aortic anatomies, the average and maximum absolute anatomical discrepancies were below 0.5 mm and 1.6 mm, respectively. Overall, the SIREN, MFN-Gabor, and MHE architectures demonstrated the best performance. Source code and data is available at https://github.com/desResLab/nrf.

Method: Integrates crystal representation learning with language understanding using a pre-trained crystal encoder with projection layer to compress structural information into token space, aligning with language instructions. Uses a curated structure-property-knowledge database.

Result: Outperforms state-of-the-art closed-source LLMs (GPT-5, Gemini-2.5-Pro, DeepSeek-R1) in property prediction and knowledge generation tasks despite using fewer parameters.

Conclusion: Multimodal approaches are crucial for porous material understanding, and L2M3OF establishes a foundation for next-generation AI systems in materials discovery.

Abstract: Large language models have demonstrated remarkable reasoning capabilities across diverse natural language tasks. However, comparable breakthroughs in scientific discovery are more limited, because understanding complex physical phenomena demands multifaceted representations far beyond language alone. A compelling example is the design of functional materials such as MOFs-critical for a range of impactful applications like carbon capture and hydrogen storage. Navigating their vast and intricate design space in language-based representations interpretable by LLMs is challenging due to the numerous possible three-dimensional atomic arrangements and strict reticular rules of coordination geometry and topology. Despite promising early results in LLM-assisted discovery for simpler materials systems, MOF design remains heavily reliant on tacit human expertise rarely codified in textual information alone. To overcome this barrier, we introduce L2M3OF, the first multimodal LLM for MOFs. L2M3OF integrates crystal representation learning with language understanding to process structural, textual, and knowledge modalities jointly. L2M3OF employs a pre-trained crystal encoder with a lightweight projection layer to compress structural information into a token space, enabling efficient alignment with language instructions. To facilitate training and evaluation, we curate a structure-property-knowledge database of crystalline materials and benchmark L2M3OF against state-of-the-art closed-source LLMs such as GPT-5, Gemini-2.5-Pro and DeepSeek-R1. Experiments show that L2M3OF outperforms leading text-based closed-source LLMs in property prediction and knowledge generation tasks, despite using far fewer parameters. These results highlight the importance of multimodal approaches for porous material understanding and establish L2M3OF as a foundation for next-generation AI systems in materials discovery.

Method: MeDyate introduces LaRa (Layer Ranking) for principled layer pre-selection and a dynamic channel sampling strategy that exploits temporal stability of channel importance distributions. It dynamically resamples channels between epochs using importance-weighted probabilities to explore parameter space while respecting memory budgets.

Result: Extensive evaluation across multiple tasks and architectures shows MeDyate achieves state-of-the-art performance under extreme memory constraints, consistently outperforming existing static and dynamic approaches while maintaining high computational efficiency.

Conclusion: MeDyate represents a significant step towards enabling efficient on-device learning by demonstrating effective fine-tuning with very low memory requirements, making it suitable for resource-constrained devices.

Abstract: On-device neural network training faces critical memory constraints that limit the adaptation of pre-trained models to downstream tasks. We present MeDyate, a theoretically-grounded framework for memory-constrained dynamic subnetwork adaptation. Our approach introduces two key innovations: LaRa (Layer Ranking), an improved layer importance metric that enables principled layer pre-selection, and a dynamic channel sampling strategy that exploits the temporal stability of channel importance distributions during fine-tuning. MeDyate dynamically resamples channels between epochs according to importance-weighted probabilities, ensuring comprehensive parameter space exploration while respecting strict memory budgets. Extensive evaluation across a large panel of tasks and architectures demonstrates that MeDyate achieves state-of-the-art performance under extreme memory constraints, consistently outperforming existing static and dynamic approaches while maintaining high computational efficiency. Our method represents a significant step towards enabling efficient on-device learning by demonstrating effective fine-tuning with memory budgets as low as a few hundred kB of RAM.

Method: Uses Grouped Lattice Vector Quantization with learnable generation matrices and Babai rounding for stable optimization during training, enabling efficient matrix-vector multiplication decoding.

Result: Achieves superior trade-off between model size and accuracy compared to existing post-training quantization baselines across multiple benchmarks.

Conclusion: GLVQ provides an effective solution for deploying large models under resource constraints while maintaining performance.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities but typically require extensive computational resources and memory for inference. Post-training quantization (PTQ) can effectively reduce these demands by storing weights in lower bit-width formats. However, standard uniform quantization often leads to notable performance degradation, particularly in low-bit scenarios. In this work, we introduce a Grouped Lattice Vector Quantization (GLVQ) framework that assigns each group of weights a customized lattice codebook, defined by a learnable generation matrix. To address the non-differentiability of the quantization process, we adopt Babai rounding to approximate nearest-lattice-point search during training, which enables stable optimization of the generation matrices. Once trained, decoding reduces to a simple matrix-vector multiplication, yielding an efficient and practical quantization pipeline. Experiments on multiple benchmarks show that our approach achieves a better trade-off between model size and accuracy compared to existing post-training quantization baselines, highlighting its effectiveness in deploying large models under stringent resource constraints. Our source code is available on GitHub repository: https://github.com/xzhang9308/GLVQ.

Method: Integrates bidirectional gated recurrent units (BiGRU) to optimize Transformer architecture, with comparative experiments against decision tree, random forest, Adaboost, and XGBoost benchmarks.

Result: Achieves lowest MSE and RMSE values, excellent MAE and R2 performance, with comprehensive predictive performance far exceeding benchmark methods.

Conclusion: The BiGRU-optimized Transformer model efficiently and accurately predicts GPU memory demand, significantly improving prediction accuracy over traditional methods, providing technical support for resource optimization.

Abstract: In response to the increasingly critical demand for accurate prediction of GPU memory resources in deep learning tasks, this paper deeply analyzes the current research status and innovatively proposes a deep learning model that integrates bidirectional gated recurrent units (BiGRU) to optimize the Transformer architecture, aiming to improve the accuracy of memory demand prediction. To verify the effectiveness of the model, a carefully designed comparative experiment was conducted, selecting four representative basic machine learning models: decision tree, random forest, Adaboost, and XGBoost as benchmarks. The detailed experimental results show that the BiGRU Transformer optimization model proposed in this paper exhibits significant advantages in key evaluation indicators: in terms of mean square error (MSE) and root mean square error (RMSE), the model achieves the lowest value among all comparison models, and its predicted results have the smallest deviation from the actual values; In terms of mean absolute error (MAE) and coefficient of determination (R2) indicators, the model also performs well and the results are balanced and stable, with comprehensive predictive performance far exceeding the benchmark machine learning methods compared. In summary, the Transformer model based on bidirectional gated recurrent unit optimization successfully constructed in this study can efficiently and accurately complete GPU memory demand prediction tasks in deep learning tasks, and its prediction accuracy has been significantly improved compared to traditional machine learning methods. This research provides strong technical support and reliable theoretical basis for optimizing resource scheduling and management of deep learning tasks, and improving the utilization efficiency of computing clusters.

Method: Augmented Lagrangian methods are applied to constrained learning problems, with theoretical analysis establishing strong duality under mild conditions and convergence proofs for dual ascent algorithms.

Result: The paper proves strong duality results, convergence to feasible and optimal primal solutions, provides PAC-style generalization guarantees, and demonstrates effectiveness on fairness-constrained classification tasks.

Conclusion: Augmented Lagrangian methods are effective for constrained learning problems, offering strong theoretical guarantees and practical performance in fairness-constrained settings.

Abstract: Despite the non-convexity of most modern machine learning parameterizations, Lagrangian duality has become a popular tool for addressing constrained learning problems. We revisit Augmented Lagrangian methods, which aim to mitigate the duality gap in non-convex settings while requiring only minimal modifications, and have remained comparably unexplored in constrained learning settings. We establish strong duality results under mild conditions, prove convergence of dual ascent algorithms to feasible and optimal primal solutions, and provide PAC-style generalization guarantees. Finally, we demonstrate its effectiveness on fairness constrained classification tasks.

Method: Uses Evolutionary Optimization of Neuromorphic Systems (EONS) algorithm to evolve sparse, stateful SNNs by jointly optimizing architectures and parameters. Inputs are encoded into spike trains, and predictions use thresholding of output neuron spike counts. Incorporates voting ensemble methods for improved performance.

Result: For radioactive source detection: SNNs with 49 neurons and 66 synapses achieved 51.8% TPR at 1/hr false alarm rate, outperforming PCA (42.7%) and deep learning (49.8%) baselines. Three-model ensemble increased TPR to 67.1%. Hardware deployment showed 2mW power consumption and 20.2ms latency. For seizure detection: ensemble achieved 95% TPR with 16% false positive rate, comparable to deep learning approaches with reduced parameters.

Conclusion: The framework successfully trains compact, efficient SNNs for time series classification tasks, demonstrating superior performance over traditional methods while maintaining low computational requirements and power consumption, with generalizability across different domains.

Abstract: We present a general framework for training spiking neural networks (SNNs) to perform binary classification on multivariate time series, with a focus on step-wise prediction and high precision at low false alarm rates. The approach uses the Evolutionary Optimization of Neuromorphic Systems (EONS) algorithm to evolve sparse, stateful SNNs by jointly optimizing their architectures and parameters. Inputs are encoded into spike trains, and predictions are made by thresholding a single output neuron’s spike counts. We also incorporate simple voting ensemble methods to improve performance and robustness. To evaluate the framework, we apply it with application-specific optimizations to the task of detecting low signal-to-noise ratio radioactive sources in gamma-ray spectral data. The resulting SNNs, with as few as 49 neurons and 66 synapses, achieve a 51.8% true positive rate (TPR) at a false alarm rate of 1/hr, outperforming PCA (42.7%) and deep learning (49.8%) baselines. A three-model any-vote ensemble increases TPR to 67.1% at the same false alarm rate. Hardware deployment on the microCaspian neuromorphic platform demonstrates 2mW power consumption and 20.2ms inference latency. We also demonstrate generalizability by applying the same framework, without domain-specific modification, to seizure detection in EEG recordings. An ensemble achieves 95% TPR with a 16% false positive rate, comparable to recent deep learning approaches with significant reduction in parameter count.

Method: Treats original AR model as teacher providing conditional scores in latent space. Uses conditional score distillation loss to train one-step generator by predicting conditional score of generated distribution at every token position.

Result: Achieves FID increase from 3.40 to 5.43 on ImageNet-256, 67% reduction in gap between one-step and original AR model compared to DD1, with 12.3x training speed-up.

Conclusion: DD2 significantly advances one-step AR generation, enabling fast and high-quality AR modeling without pre-defined mapping constraints.

Abstract: Image Auto-regressive (AR) models have emerged as a powerful paradigm of visual generative models. Despite their promising performance, they suffer from slow generation speed due to the large number of sampling steps required. Although Distilled Decoding 1 (DD1) was recently proposed to enable few-step sampling for image AR models, it still incurs significant performance degradation in the one-step setting, and relies on a pre-defined mapping that limits its flexibility. In this work, we propose a new method, Distilled Decoding 2 (DD2), to further advances the feasibility of one-step sampling for image AR models. Unlike DD1, DD2 does not without rely on a pre-defined mapping. We view the original AR model as a teacher model which provides the ground truth conditional score in the latent embedding space at each token position. Based on this, we propose a novel \emph{conditional score distillation loss} to train a one-step generator. Specifically, we train a separate network to predict the conditional score of the generated distribution and apply score distillation at every token position conditioned on previous tokens. Experimental results show that DD2 enables one-step sampling for image AR models with an minimal FID increase from 3.40 to 5.43 on ImageNet-256. Compared to the strongest baseline DD1, DD2 reduces the gap between the one-step sampling and original AR model by 67%, with up to 12.3$\times$ training speed-up simultaneously. DD2 takes a significant step toward the goal of one-step AR generation, opening up new possibilities for fast and high-quality AR modeling. Code is available at https://github.com/imagination-research/Distilled-Decoding-2.

Method: Proposes FRG framework using optimized adversarial model to provide high-confidence fairness guarantees with user-defined error thresholds.

Result: FRG consistently bounds unfairness across downstream models and tasks, outperforming six state-of-the-art fair representation learning methods on three real-world datasets.

Conclusion: FRG successfully provides high-confidence fairness guarantees in representation learning, ensuring bounded demographic disparity in downstream predictions.

Abstract: Representation learning is increasingly applied to generate representations that generalize well across multiple downstream tasks. Ensuring fairness guarantees in representation learning is crucial to prevent unfairness toward specific demographic groups in downstream tasks. In this work, we formally introduce the task of learning representations that achieve high-confidence fairness. We aim to guarantee that demographic disparity in every downstream prediction remains bounded by a user-defined error threshold $\epsilon$, with controllable high probability. To this end, we propose the Fair Representation learning with high-confidence Guarantees (FRG) framework, which provides these high-confidence fairness guarantees by leveraging an optimized adversarial model. We empirically evaluate FRG on three real-world datasets, comparing its performance to six state-of-the-art fair representation learning methods. Our results demonstrate that FRG consistently bounds unfairness across a range of downstream models and tasks.

Method: Propose ZO-FC approach that applies ZO optimization to adapter-based PEFT modules while using FO optimization for classifiers, combining stability benefits of ZO with adaptability of FO.

Result: ZO optimization enhances stability but reduces plasticity, particularly with learnable classifiers. ZO-FC achieves effective balance between stability and plasticity with negligible memory overhead.

Conclusion: ZO-FC provides a practical memory-efficient solution for on-device continual learning by leveraging ZO optimization’s stability benefits while preserving FO optimization’s plasticity.

Abstract: Zeroth-order (ZO) optimization has gained attention as a memory-efficient alternative to first-order (FO) methods, particularly in settings where gradient computation is expensive or even impractical. Beyond its memory efficiency, in this work, we investigate ZO optimization for continual learning (CL) as a novel approach to address the plasticity-stability-efficiency trilemma. Through theoretical analysis and empirical evidence, we show that ZO optimization naturally leads to flatter loss landscapes, which in turn reduce forgetting in CL. However, this stability comes at a cost of plasticity: due to its imprecise gradient estimates and slower convergence, ZO optimization tends to be less effective than FO in acquiring new task-specific knowledge, particularly under constrained training budgets. To better understand this trade-off, we conduct a holistic evaluation of ZO optimization applied to various existing CL methods. Our findings reveal that ZO optimization enhances stability but often undermines plasticity, particularly when used with learnable classifiers. Motivated by this insight, we propose ZO-FC, a simple but effective approach that applies ZO optimization to a single adapter-based PEFT module with FO optimized classifier. This design leverages the stability benefits of ZO while preserving the adaptability of FO updates with negligible memory overhead. Experiments demonstrate that ZO-FC achieves an effective balance between stability and plasticity, offering a practical and memory-efficient solution for on-device CL.

Method: The authors develop a theoretical framework covering gradient-based algorithms including one-pass SGD, SGD with batch reuse, and introduce a new layer-wise training algorithm using two-timescales approach with different learning rates per layer.

Result: The study demonstrates a phase transition: small learning rates operate in the information exponent regime while large learning rates shift to the generative exponent regime, which can have much smaller sample complexity.

Conclusion: The choice of learning rate is as crucial as algorithm design for achieving statistical and computational efficiency in neural network training, with different regimes offering distinct sample complexity benefits.

Abstract: To understand feature learning dynamics in neural networks, recent theoretical works have focused on gradient-based learning of Gaussian single-index models, where the label is a nonlinear function of a latent one-dimensional projection of the input. While the sample complexity of online SGD is determined by the information exponent of the link function, recent works improved this by performing multiple gradient steps on the same sample with different learning rates – yielding a non-correlational update rule – and instead are limited by the (potentially much smaller) generative exponent. However, this picture is only valid when these learning rates are sufficiently large. In this paper, we characterize the relationship between learning rate(s) and sample complexity for a broad class of gradient-based algorithms that encapsulates both correlational and non-correlational updates. We demonstrate that, in certain cases, there is a phase transition from an “information exponent regime” with small learning rate to a “generative exponent regime” with large learning rate. Our framework covers prior analyses of one-pass SGD and SGD with batch reuse, while also introducing a new layer-wise training algorithm that leverages a two-timescales approach (via different learning rates for each layer) to go beyond correlational queries without reusing samples or modifying the loss from squared error. Our theoretical study demonstrates that the choice of learning rate is as important as the design of the algorithm in achieving statistical and computational efficiency.

Method: CIPHER integrates iSAX for interpretable compression and indexing, HDBSCAN for clustering recurring phenomena, and human-in-the-loop validation where domain experts label representative samples and annotations are propagated across clusters.

Result: The framework successfully recovers meaningful solar wind phenomena (coronal mass ejections and stream interaction regions) in OMNI data, demonstrating effective classification in space weather research.

Conclusion: CIPHER presents a general strategy combining symbolic representations, unsupervised learning, and expert knowledge to address label scarcity in time series across physical sciences, with publicly available code for reproducibility.

Abstract: Labeling or classifying time series is a persistent challenge in the physical sciences, where expert annotations are scarce, costly, and often inconsistent. Yet robust labeling is essential to enable machine learning models for understanding, prediction, and forecasting. We present the \textit{Clustering and Indexation Pipeline with Human Evaluation for Recognition} (CIPHER), a framework designed to accelerate large-scale labeling of complex time series in physics. CIPHER integrates \textit{indexable Symbolic Aggregate approXimation} (iSAX) for interpretable compression and indexing, density-based clustering (HDBSCAN) to group recurring phenomena, and a human-in-the-loop step for efficient expert validation. Representative samples are labeled by domain scientists, and these annotations are propagated across clusters to yield systematic, scalable classifications. We evaluate CIPHER on the task of classifying solar wind phenomena in OMNI data, a central challenge in space weather research, showing that the framework recovers meaningful phenomena such as coronal mass ejections and stream interaction regions. Beyond this case study, CIPHER highlights a general strategy for combining symbolic representations, unsupervised learning, and expert knowledge to address label scarcity in time series across the physical sciences. The code and configuration files used in this study are publicly available to support reproducibility.

Method: Two-stage framework: PCNO uses physics-consistent projection layer in Fourier space to enforce mass/momentum conservation; DiffPCNO adds consistency model-based diffusion for uncertainty quantification.

Result: Achieves high-fidelity spatiotemporal predictions while preserving physical consistency and uncertainty across diverse systems (turbulent flow, flood/atmospheric forecasting) and spatial resolutions.

Conclusion: Provides robust and versatile approach for accurate, physically grounded, and uncertainty-aware spatiotemporal forecasting that outperforms existing methods.

Abstract: Accurate long-term forecasting of spatiotemporal dynamics remains a fundamental challenge across scientific and engineering domains. Existing machine learning methods often neglect governing physical laws and fail to quantify inherent uncertainties in spatiotemporal predictions. To address these challenges, we introduce a physics-consistent neural operator (PCNO) that enforces physical constraints by projecting surrogate model outputs onto function spaces satisfying predefined laws. A physics-consistent projection layer within PCNO efficiently computes mass and momentum conservation in Fourier space. Building upon deterministic predictions, we further propose a diffusion model-enhanced PCNO (DiffPCNO), which leverages a consistency model to quantify and mitigate uncertainties, thereby improving the accuracy and reliability of forecasts. PCNO and DiffPCNO achieve high-fidelity spatiotemporal predictions while preserving physical consistency and uncertainty across diverse systems and spatial resolutions, ranging from turbulent flow modeling to real-world flood/atmospheric forecasting. Our two-stage framework provides a robust and versatile approach for accurate, physically grounded, and uncertainty-aware spatiotemporal forecasting.

Method: Used the 52-hour LibriBrain corpus with standardized train/validation/test splits. Implemented a compact 1-D Conv/ResNet baseline with focal loss and top-k pooling, trainable on consumer GPUs.

Result: The reference model achieved ~13x the permutation baseline AUPRC on held-out sessions. Performance scales log-linearly with training hours, and word-level factors (frequency, duration) systematically affect detectability.

Conclusion: KWS is a viable intermediate task for BCIs, with predictable scaling and identifiable word-level factors influencing performance. The framework enables reproducible benchmarking and community experimentation.

Abstract: Non-invasive brain-computer interfaces (BCIs) are beginning to benefit from large, public benchmarks. However, current benchmarks target relatively simple, foundational tasks like Speech Detection and Phoneme Classification, while application-ready results on tasks like Brain-to-Text remain elusive. We propose Keyword Spotting (KWS) as a practically applicable, privacy-aware intermediate task. Using the deep 52-hour, within-subject LibriBrain corpus, we provide standardized train/validation/test splits for reproducible benchmarking, and adopt an evaluation protocol tailored to extreme class imbalance. Concretely, we use area under the precision-recall curve (AUPRC) as a robust evaluation metric, complemented by false alarms per hour (FA/h) at fixed recall to capture user-facing trade-offs. To simplify deployment and further experimentation within the research community, we are releasing an updated version of the pnpl library with word-level dataloaders and Colab-ready tutorials. As an initial reference model, we present a compact 1-D Conv/ResNet baseline with focal loss and top-k pooling that is trainable on a single consumer-class GPU. The reference model achieves approximately 13x the permutation baseline AUPRC on held-out sessions, demonstrating the viability of the task. Exploratory analyses reveal: (i) predictable within-subject scaling - performance improves log-linearly with more training hours - and (ii) the existence of word-level factors (frequency and duration) that systematically modulate detectability.

Method: Uses a generative model conditioned on user preferences via preference direction vectors, employs class probability estimators to predict non-dominance relations and implicitly estimate probability of hypervolume improvement, and updates the model iteratively using Pareto membership and preference alignment.

Result: Achieves strong sample efficiency on synthetic benchmarks and protein design tasks, produces high-quality Pareto set approximations, and effectively incorporates user preferences without requiring retraining for different preference settings.

Conclusion: A-GPS provides a simple yet powerful approach for multi-objective optimization that flexibly captures user preferences, avoids explicit hypervolume computation, and generates amortized generative models capable of sampling across the Pareto front efficiently.

Abstract: We introduce active generation of Pareto sets (A-GPS), a new framework for online discrete black-box multi-objective optimization (MOO). A-GPS learns a generative model of the Pareto set that supports a-posteriori conditioning on user preferences. The method employs a class probability estimator (CPE) to predict non-dominance relations and to condition the generative model toward high-performing regions of the search space. We also show that this non-dominance CPE implicitly estimates the probability of hypervolume improvement (PHVI). To incorporate subjective trade-offs, A-GPS introduces preference direction vectors that encode user-specified preferences in objective space. At each iteration, the model is updated using both Pareto membership and alignment with these preference directions, producing an amortized generative model capable of sampling across the Pareto front without retraining. The result is a simple yet powerful approach that achieves high-quality Pareto set approximations, avoids explicit hypervolume computation, and flexibly captures user preferences. Empirical results on synthetic benchmarks and protein design tasks demonstrate strong sample efficiency and effective preference incorporation.

Method: Developed a randomized algorithm using relax-and-round framework with resource reservation (set-aside mechanism) for fairness, plus a learning-augmented variant incorporating ML predictions.

Result: The algorithm ensures integral solutions achieve same expected performance as fractional solutions while preserving fairness guarantees across multiple classes.

Conclusion: The proposed approach successfully bridges fairness and efficiency in online multi-class selection, with learning augmentation providing practical balance in real-world settings.

Abstract: We study the online multi-class selection problem with group fairness guarantees, where limited resources must be allocated to sequentially arriving agents. Our work addresses two key limitations in the existing literature. First, we introduce a novel lossless rounding scheme that ensures the integral algorithm achieves the same expected performance as any fractional solution. Second, we explicitly address the challenges introduced by agents who belong to multiple classes. To this end, we develop a randomized algorithm based on a relax-and-round framework. The algorithm first computes a fractional solution using a resource reservation approach – referred to as the set-aside mechanism – to enforce fairness across classes. The subsequent rounding step preserves these fairness guarantees without degrading performance. Additionally, we propose a learning-augmented variant that incorporates untrusted machine-learned predictions to better balance fairness and efficiency in practical settings.

Method: The authors formalize a differential privacy definition tailored to policy optimization, then systematically analyze sample complexity of algorithms like policy gradient and natural policy gradient under DP constraints using a unified framework.

Result: Theoretical results show privacy costs often appear as lower-order terms in sample complexity, revealing subtle but important observations about private policy optimization settings.

Conclusion: The analysis provides valuable practical insights for developing privacy-preserving policy optimization algorithms in sensitive applications.

Abstract: Policy optimization (PO) is a cornerstone of modern reinforcement learning (RL), with diverse applications spanning robotics, healthcare, and large language model training. The increasing deployment of PO in sensitive domains, however, raises significant privacy concerns. In this paper, we initiate a theoretical study of differentially private policy optimization, focusing explicitly on its sample complexity. We first formalize an appropriate definition of differential privacy (DP) tailored to PO, addressing the inherent challenges arising from on-policy learning dynamics and the subtlety involved in defining the unit of privacy. We then systematically analyze the sample complexity of widely-used PO algorithms, including policy gradient (PG), natural policy gradient (NPG) and more, under DP constraints and various settings, via a unified framework. Our theoretical results demonstrate that privacy costs can often manifest as lower-order terms in the sample complexity, while also highlighting subtle yet important observations in private PO settings. These offer valuable practical insights for privacy-preserving PO algorithms.

Method: Leverages Dask for large-scale statistical computations and Kernel Density Matrices (KDM) to estimate univariate and bivariate distributions of solar wind parameters and anomaly thresholds.

Result: Revealed characteristic trends including increasing solar wind speed with distance from the Sun, decreasing proton density, and the inverse relationship between speed and density. Provided quantitative insights into solar wind processes.

Conclusion: Offers a tractable, interpretable, and distributed methodology for exploring complex physical datasets and facilitates reproducible analysis of large-scale in situ measurements. Processed data products and tools are publicly available to advance solar wind studies.

Abstract: We present a scalable machine learning framework for analyzing Parker Solar Probe (PSP) solar wind data using distributed processing and the quantum-inspired Kernel Density Matrices (KDM) method. The PSP dataset (2018–2024) exceeds 150 GB, challenging conventional analysis approaches. Our framework leverages Dask for large-scale statistical computations and KDM to estimate univariate and bivariate distributions of key solar wind parameters, including solar wind speed, proton density, and proton thermal speed, as well as anomaly thresholds for each parameter. We reveal characteristic trends in the inner heliosphere, including increasing solar wind speed with distance from the Sun, decreasing proton density, and the inverse relationship between speed and density. Solar wind structures play a critical role in enhancing and mediating extreme space weather phenomena and can trigger geomagnetic storms; our analyses provide quantitative insights into these processes. This approach offers a tractable, interpretable, and distributed methodology for exploring complex physical datasets and facilitates reproducible analysis of large-scale in situ measurements. Processed data products and analysis tools are made publicly available to advance future studies of solar wind dynamics and space weather forecasting. The code and configuration files used in this study are publicly available to support reproducibility.

Method: Propose selecting the shortest solution as a simple heuristic, based on the observation that models operate in concise confident and verbose uncertain regimes.

Result: The shortest-answer heuristic is competitive with complex methods like self-consistency across benchmarks while significantly reducing computational overhead.

Conclusion: Shortest-answer selection provides a Pareto improvement over self-consistency and works even when output equality is not well-defined.

Abstract: Reasoning models represent a significant advance in LLM capabilities, particularly for complex reasoning tasks such as mathematics and coding. Previous studies confirm that parallel test-time compute-sampling multiple solutions and selecting the best one-can further enhance the predictive performance of LLMs. However, strategies in this area often require complex scoring, thus increasing computational cost and complexity. In this work, we demonstrate that the simple and counterintuitive heuristic of selecting the shortest solution is highly effective. We posit that the observed effectiveness stems from models operating in two distinct regimes: a concise, confident conventional regime and a verbose overthinking regime characterized by uncertainty, and we show evidence of a critical point where the overthinking regime begins to be significant. By selecting the shortest answer, the heuristic preferentially samples from the conventional regime. We confirm that this approach is competitive with more complex methods such as self-consistency across two challenging benchmarks while significantly reducing computational overhead. The shortest-answer heuristic provides a Pareto improvement over self-consistency and applies even to tasks where output equality is not well defined.

Method: Analyze gradient flow on a two-layer ReLU network for classifying orthogonally separable data, studying the training dynamics and implicit bias.

Result: The authors prove that Neural Collapse provably emerges in this setting, showing how data structure and nonlinear activations affect NC characterizations.

Conclusion: The work advances theoretical understanding of NC by revealing the role of implicit bias in training dynamics and demonstrating NC emergence under more realistic conditions with data structure constraints.

Abstract: Among many mysteries behind the success of deep networks lies the exceptional discriminative power of their learned representations as manifested by the intriguing Neural Collapse (NC) phenomenon, where simple feature structures emerge at the last layer of a trained neural network. Prior works on the theoretical understandings of NC have focused on analyzing the optimization landscape of matrix-factorization-like problems by considering the last-layer features as unconstrained free optimization variables and showing that their global minima exhibit NC. In this paper, we show that gradient flow on a two-layer ReLU network for classifying orthogonally separable data provably exhibits NC, thereby advancing prior results in two ways: First, we relax the assumption of unconstrained features, showing the effect of data structure and nonlinear activations on NC characterizations. Second, we reveal the role of the implicit bias of the training dynamics in facilitating the emergence of NC.

Method: Uses OpenCL fine-grained shared virtual memory (SVM) for lightweight synchronization and machine learning models to predict execution times of tasks on CPU and GPU, accounting for GPU kernel performance characteristics and dispatch times.

Result: Achieved up to 1.89x speedup for linear layers and 1.75x speedup for convolutional layers on four mobile platforms, close to the theoretical maximums of 2.01x and 1.87x respectively.

Conclusion: The proposed CPU-GPU co-execution strategy with lightweight synchronization and accurate execution time prediction effectively reduces inference latency in mobile deep neural networks.

Abstract: Deploying deep neural networks on mobile devices is increasingly important but remains challenging due to limited computing resources. On the other hand, their unified memory architecture and narrower gap between CPU and GPU performance provide an opportunity to reduce inference latency by assigning tasks to both CPU and GPU. The main obstacles for such collaborative execution are the significant synchronization overhead required to combine partial results, and the difficulty of predicting execution times of tasks assigned to CPU and GPU (due to the dynamic selection of implementations and parallelism level). To overcome these obstacles, we propose both a lightweight synchronization mechanism based on OpenCL fine-grained shared virtual memory (SVM) and machine learning models to accurately predict execution times. Notably, these models capture the performance characteristics of GPU kernels and account for their dispatch times. A comprehensive evaluation on four mobile platforms shows that our approach can quickly select CPU-GPU co-execution strategies achieving up to 1.89x speedup for linear layers and 1.75x speedup for convolutional layers (close to the achievable maximum values of 2.01x and 1.87x, respectively, found by exhaustive grid search on a Pixel~5 smartphone).

Method: Uses two key modules: DePE decomposes model weights into static dictionary and updatable lookup table (only latter encrypted), and PrME applies encryption-aware pruning to minimize encrypted parameters using history-guided masks.

Result: Reduces communication cost by 402-748× and accelerates training by 28-65× compared to fully encrypted FL, while outperforming state-of-the-art selective encryption methods by 51-155× in overhead and 4-19× in speed.

Conclusion: DictPFL demonstrates that HE-based private federated learning is practical for real-world deployment, with runtime within 2× of plaintext FL.

Abstract: Federated Learning (FL) enables collaborative model training across institutions without sharing raw data. However, gradient sharing still risks privacy leakage, such as gradient inversion attacks. Homomorphic Encryption (HE) can secure aggregation but often incurs prohibitive computational and communication overhead. Existing HE-based FL methods sit at two extremes: encrypting all gradients for full privacy at high cost, or partially encrypting gradients to save resources while exposing vulnerabilities. We present DictPFL, a practical framework that achieves full gradient protection with minimal overhead. DictPFL encrypts every transmitted gradient while keeping non-transmitted parameters local, preserving privacy without heavy computation. It introduces two key modules: Decompose-for-Partial-Encrypt (DePE), which decomposes model weights into a static dictionary and an updatable lookup table, only the latter is encrypted and aggregated, while the static dictionary remains local and requires neither sharing nor encryption; and Prune-for-Minimum-Encrypt (PrME), which applies encryption-aware pruning to minimize encrypted parameters via consistent, history-guided masks. Experiments show that DictPFL reduces communication cost by 402-748$\times$ and accelerates training by 28-65$\times$ compared to fully encrypted FL, while outperforming state-of-the-art selective encryption methods by 51-155$\times$ in overhead and 4-19$\times$ in speed. Remarkably, DictPFL’s runtime is within 2$\times$ of plaintext FL, demonstrating for the first time, that HE-based private federated learning is practical for real-world deployment. The code is publicly available at https://github.com/UCF-ML-Research/DictPFL.

Method: Creates a tri-partite heterogeneous graph with motif, molecule, and property nodes at global level for long-range patterns, and builds local subgraphs for each node to focus on informative neighbors.

Result: Outperforms state-of-the-art methods on five standard FSMPP benchmarks, demonstrating consistent performance improvements.

Conclusion: Integrating global motif knowledge with fine-grained local context effectively advances robust few-shot molecular property prediction.

Abstract: Molecular property prediction (MPP) is a cornerstone of drug discovery and materials science, yet conventional deep learning approaches depend on large labeled datasets that are often unavailable. Few-shot Molecular property prediction (FSMPP) addresses this scarcity by incorporating relational inductive bias through a context graph that links molecule nodes to property nodes, but such molecule-property graphs offer limited structural guidance. We propose a comprehensive solution: Motif Driven Global-Local Context Graph for few-shot molecular property prediction, which enriches contextual information at both the global and local levels. At the global level, chemically meaningful motif nodes representing shared substructures, such as rings or functional groups, are introduced to form a global tri-partite heterogeneous graph, yielding motif-molecule-property connections that capture long-range compositional patterns and enable knowledge transfer among molecules with common motifs. At the local level, we build a subgraph for each node in the molecule-property pair and encode them separately to concentrate the model’s attention on the most informative neighboring molecules and motifs. Experiments on five standard FSMPP benchmarks demonstrate that our framework consistently outperforms state-of-the-art methods. These results underscore the effectiveness of integrating global motif knowledge with fine-grained local context to advance robust few-shot molecular property prediction.

Method: ESCORT extends Stein Variational Gradient Descent (SVGD) with two key innovations: correlation-aware projections that model dependencies between state dimensions, and temporal consistency constraints that stabilize updates while preserving correlation structures.

Result: ESCORT consistently outperforms state-of-the-art methods in belief approximation accuracy and downstream decision quality across POMDP domains and synthetic multi-modal distributions of varying dimensionality.

Conclusion: ESCORT provides an effective solution for maintaining representational accuracy in complex POMDP environments by dynamically adapting to belief landscape complexity without resampling or restrictive distributional assumptions.

Abstract: In Partially Observable Markov Decision Processes (POMDPs), maintaining and updating belief distributions over possible underlying states provides a principled way to summarize action-observation history for effective decision-making under uncertainty. As environments grow more realistic, belief distributions develop complexity that standard mathematical models cannot accurately capture, creating a fundamental challenge in maintaining representational accuracy. Despite advances in deep learning and probabilistic modeling, existing POMDP belief approximation methods fail to accurately represent complex uncertainty structures such as high-dimensional, multi-modal belief distributions, resulting in estimation errors that lead to suboptimal agent behaviors. To address this challenge, we present ESCORT (Efficient Stein-variational and sliced Consistency-Optimized Representation for Temporal beliefs), a particle-based framework for capturing complex, multi-modal distributions in high-dimensional belief spaces. ESCORT extends SVGD with two key innovations: correlation-aware projections that model dependencies between state dimensions, and temporal consistency constraints that stabilize updates while preserving correlation structures. This approach retains SVGD’s attractive-repulsive particle dynamics while enabling accurate modeling of intricate correlation patterns. Unlike particle filters prone to degeneracy or parametric methods with fixed representational capacity, ESCORT dynamically adapts to belief landscape complexity without resampling or restrictive distributional assumptions. We demonstrate ESCORT’s effectiveness through extensive evaluations on both POMDP domains and synthetic multi-modal distributions of varying dimensionality, where it consistently outperforms state-of-the-art methods in terms of belief approximation accuracy and downstream decision quality.

Method: Frames feature selection as continuous relaxation using a noising mechanism, optimizing over variance of Bayes-optimal predictor without backpropagation through training.

Result: Validated through experiments on synthetic and real-world datasets.

Conclusion: Developed a model-agnostic framework that balances downstream prediction performance across populations while minimizing feature collection costs.

Abstract: We study the problem of selecting limited features to observe such that models trained on them can perform well simultaneously across multiple subpopulations. This problem has applications in settings where collecting each feature is costly, e.g. requiring adding survey questions or physical sensors, and we must be able to use the selected features to create high-quality downstream models for different populations. Our method frames the problem as a continuous relaxation of traditional variable selection using a noising mechanism, without requiring backpropagation through model training processes. By optimizing over the variance of a Bayes-optimal predictor, we develop a model-agnostic framework that balances overall performance of downstream prediction across populations. We validate our approach through experiments on both synthetic datasets and real-world data.

Method: Augments standard RL loss with additional loss that uses learned proposals to guide sampling of low-reward outputs, then reduces those outputs’ probability.

Result: RePULSe produces better tradeoff between expected reward and probability of undesired outputs, and is more adversarially robust compared to standard RL alignment approaches and alternatives.

Conclusion: RePULSe effectively addresses the limitation of standard RL methods by providing improved performance in reducing undesirable outputs while maintaining good average reward performance.

Abstract: Reinforcement learning (RL) has become a predominant technique to align language models (LMs) with human preferences or promote outputs which are deemed to be desirable by a given reward function. Standard RL approaches optimize average reward, while methods explicitly focused on reducing the probability of undesired outputs typically come at a cost to average-case performance. To improve this tradeoff, we introduce RePULSe, a new training method that augments the standard RL loss with an additional loss that uses learned proposals to guide sampling low-reward outputs, and then reduces those outputs’ probability. We run experiments demonstrating that RePULSe produces a better tradeoff of expected reward versus the probability of undesired outputs and is more adversarially robust, compared to standard RL alignment approaches and alternatives.

Method: Models aggregated power output as composite of small grid outputs, using unit output function multiplied by capacity. Proposes efficient algorithms with upper-bound approximation to overcome computational bottlenecks.

Result: Validated through deployment across various Chinese cities, significantly reducing potential losses and providing valuable grid operation insights.

Conclusion: SolarBoost demonstrates superiority of grid-level modeling through theoretical analysis and experiments, offering an effective solution for DPV power forecasting.

Abstract: This paper presents SolarBoost, a novel approach for forecasting power output in distributed photovoltaic (DPV) systems. While existing centralized photovoltaic (CPV) methods are able to precisely model output dependencies due to uniformity, it is difficult to apply such techniques to DPV systems, as DPVs face challenges such as missing grid-level data, temporal shifts in installed capacity, geographic variability, and panel diversity. SolarBoost overcomes these challenges by modeling aggregated power output as a composite of output from small grids, where each grid output is modeled using a unit output function multiplied by its capacity. This approach decouples the homogeneous unit output function from dynamic capacity for accurate prediction. Efficient algorithms over an upper-bound approximation are proposed to overcome computational bottlenecks in loss functions. We demonstrate the superiority of grid-level modeling via theoretical analysis and experiments. SolarBoost has been validated through deployment across various cities in China, significantly reducing potential losses and provides valuable insights for the operation of power grids. The code for this work is available at https://github.com/DAMO-DI-ML/SolarBoost.

Method: Two-tier DDPG framework with global controller for layer selection (edge/fog/cloud) and local controllers for node assignment within chosen layers.

Result: Superior performance over baselines, especially as workflow complexity increases, demonstrating effective long-term strategy learning.

Conclusion: The framework effectively handles complex, large-scale MIoT scheduling scenarios by learning hierarchical scheduling strategies.

Abstract: The Medical Internet of Things (MIoT) demands stringent end-to-end latency guarantees for sequential healthcare workflows deployed over heterogeneous cloud-fog-edge infrastructures. Scheduling these sequential workflows to minimize makespan is an NP-hard problem. To tackle this challenge, we propose a Two-tier DDPG-based scheduling framework that decomposes the scheduling decision into a hierarchical process: a global controller performs layer selection (edge, fog, or cloud), while specialized local controllers handle node assignment within the chosen layer. The primary optimization objective is the minimization of the workflow makespan. Experiments results validate our approach, demonstrating increasingly superior performance over baselines as workflow complexity rises. This trend highlights the frameworks ability to learn effective long-term strategies, which is critical for complex, large-scale MIoT scheduling scenarios.

Method: Two-phase approach: 1) Erasure phase resets neurons in last layers to disrupt memorization, 2) Repair phase fine-tunes model while avoiding sensitive information memorization.

Result: Significantly reduces regurgitation of sensitive information with only few additional unlearning epochs, effectively sanitizing models trained on medical and confidential data.

Conclusion: SANi provides efficient model sanitization without costly retraining, particularly valuable for organizations like hospitals that have invested in training models on sensitive datasets.

Abstract: Pre-trained large language models (LLMs) are becoming useful for various tasks. To improve their performance on certain tasks, it is necessary to fine-tune them on specific data corpora (e.g., medical reports, business data). These specialized data corpora may contain sensitive data (e.g., personal or confidential data) that will be memorized by the model and likely to be regurgitated during its subsequent use. This memorization of sensitive information by the model poses a significant privacy or confidentiality issue. To remove this memorization and sanitize the model without requiring costly additional fine-tuning on a secured data corpus, we propose SANI. SANI is an unlearning approach to sanitize language models. It relies on both an erasure and repair phases that 1) reset certain neurons in the last layers of the model to disrupt the memorization of fine-grained information, and then 2) fine-tune the model while avoiding memorizing sensitive information. We comprehensively evaluate SANI to sanitize both a model fine-tuned and specialized with medical data by removing directly and indirectly identifiers from the memorization of the model, and a standard pre-trained model by removing specific terms defined as confidential information from the model. Results show that with only few additional epochs of unlearning, the model is sanitized and the number of regurgitations is drastically reduced. This approach can be particularly useful for hospitals or other industries that have already spent significant resources training models on large datasets and wish to sanitize them before sharing.

Method: Uses uncertainty-aware Reinforcement Learning with surrogate models that estimate predictive uncertainty to dynamically shape reward functions and balance multiple optimization objectives.

Result: Outperforms baselines across three benchmark datasets and multiple diffusion model architectures for molecular quality and property optimization. MD simulations and ADMET profiling show promising drug-like behavior comparable to known EGFR inhibitors.

Conclusion: Demonstrates strong potential of RL-guided generative diffusion models for advancing automated molecular design.

Abstract: Designing de novo 3D molecules with desirable properties remains a fundamental challenge in drug discovery and molecular engineering. While diffusion models have demonstrated remarkable capabilities in generating high-quality 3D molecular structures, they often struggle to effectively control complex multi-objective constraints critical for real-world applications. In this study, we propose an uncertainty-aware Reinforcement Learning (RL) framework to guide the optimization of 3D molecular diffusion models toward multiple property objectives while enhancing the overall quality of the generated molecules. Our method leverages surrogate models with predictive uncertainty estimation to dynamically shape reward functions, facilitating balance across multiple optimization objectives. We comprehensively evaluate our framework across three benchmark datasets and multiple diffusion model architectures, consistently outperforming baselines for molecular quality and property optimization. Additionally, Molecular Dynamics (MD) simulations and ADMET profiling of top generated candidates indicate promising drug-like behavior and binding stability, comparable to known Epidermal Growth Factor Receptor (EGFR) inhibitors. Our results demonstrate the strong potential of RL-guided generative diffusion models for advancing automated molecular design.

Method: Integrates Fair PCA to project CLIP-based input embeddings into a subspace that minimizes group-specific information while preserving semantic content, with empirical noise injection and unified cross-demographic projection for multi-attribute debiasing.

Result: Extensive experiments show FairImagen significantly improves fairness across gender, race, and intersectional settings with moderate trade-offs in image quality and prompt fidelity, outperforming existing post-hoc methods.

Conclusion: FairImagen provides a simple, scalable, and model-agnostic solution for equitable text-to-image generation without requiring model modifications or retraining.

Abstract: Text-to-image diffusion models, such as Stable Diffusion, have demonstrated remarkable capabilities in generating high-quality and diverse images from natural language prompts. However, recent studies reveal that these models often replicate and amplify societal biases, particularly along demographic attributes like gender and race. In this paper, we introduce FairImagen (https://github.com/fuzihaofzh/FairImagen), a post-hoc debiasing framework that operates on prompt embeddings to mitigate such biases without retraining or modifying the underlying diffusion model. Our method integrates Fair Principal Component Analysis to project CLIP-based input embeddings into a subspace that minimizes group-specific information while preserving semantic content. We further enhance debiasing effectiveness through empirical noise injection and propose a unified cross-demographic projection method that enables simultaneous debiasing across multiple demographic attributes. Extensive experiments across gender, race, and intersectional settings demonstrate that FairImagen significantly improves fairness with a moderate trade-off in image quality and prompt fidelity. Our framework outperforms existing post-hoc methods and offers a simple, scalable, and model-agnostic solution for equitable text-to-image generation.

Method: Reformulate crisp k-means and fuzzy c-means clustering as matrix factorization problems, then propose robust variants by replacing Frobenius norm with l1,2-norm. Develop alternating minimization algorithms for standard formulations and IRLS-based algorithms for robust counterparts.

Result: Successfully established matrix factorization interpretations for both crisp k-means and fuzzy c-means clustering, developed convergent algorithms for all variants, with robust formulations showing improved outlier resistance.

Conclusion: The unified matrix factorization framework provides a principled approach to extend classical clustering methods to robust variants, with proven convergence guarantees for all developed algorithms.

Abstract: This paper presents a unified matrix factorization framework for classical and robust clustering. We begin by revisiting the well-known equivalence between crisp k-means clustering and matrix factorization, following and rigorously rederiving an unpublished formulation by Bauckhage. Extending this framework, we derive an analogous matrix factorization interpretation for fuzzy c-means clustering, which to the best of our knowledge has not been previously formalized. These reformulations allow both clustering paradigms to be expressed as optimization problems over factor matrices, thereby enabling principled extensions to robust variants. To address sensitivity to outliers, we propose robust formulations for both crisp and fuzzy clustering by replacing the Frobenius norm with the l1,2-norm, which penalizes the sum of Euclidean norms across residual columns. We develop alternating minimization algorithms for the standard formulations and IRLS-based algorithms for the robust counterparts. All algorithms are theoretically proven to converge to a local minimum.

Method: Proposed system collects open weather data into a NoSQL database, fuses data at different temporal/spatial aggregation levels, and uses univariate/multivariate approaches with forecasting from neighbor stations for missing data.

Result: System achieved normalized mean squared error of 0.00013 and directional symmetry of 0.84. Experts rated system positively (3+ on 1-5 scale for all aspects except graphic design).

Conclusion: The promising preliminary results demonstrate system validity and encourage continued work in this area for weather data analysis and prediction.

Abstract: The Meteorology is a field where huge amounts of data are generated, mainly collected by sensors at weather stations, where different variables can be measured. Those data have some particularities such as high volume and dimensionality, the frequent existence of missing values in some stations, and the high correlation between collected variables. In this regard, it is crucial to make use of Big Data and Data Mining techniques to deal with those data and extract useful knowledge from them that can be used, for instance, to predict weather phenomena. In this paper, we propose a visual big data system that is designed to deal with high amounts of weather-related data and lets the user analyze those data to perform predictive tasks over the considered variables (temperature and rainfall). The proposed system collects open data and loads them onto a local NoSQL database fusing them at different levels of temporal and spatial aggregation in order to perform a predictive analysis using univariate and multivariate approaches as well as forecasting based on training data from neighbor stations in cases with high rates of missing values. The system has been assessed in terms of usability and predictive performance, obtaining an overall normalized mean squared error value of 0.00013, and an overall directional symmetry value of nearly 0.84. Our system has been rated positively by a group of experts in the area (all aspects of the system except graphic desing were rated 3 or above in a 1-5 scale). The promising preliminary results obtained demonstrate the validity of our system and invite us to keep working on this area.

Method: Uses modern machine learning techniques for bilevel optimization with implicit differentiation and conjugate gradients to compute hypergradients through Hessian-vector products in a matrix-free, variance-reduced manner.

Result: Successfully recovers known analytical optima in benchmark CARA-Normal environments and converges reliably from random initialization. Extends to complex nonlinear contracts like sigmoidal wage schedules, relative-performance compensation, multi-task contracts, and CARA-Poisson models.

Conclusion: Provides a new computational tool for contract design that enables systematic study of analytically intractable models, removing reliance on closed-form solutions.

Abstract: We study a bilevel \emph{max-max} optimization framework for principal-agent contract design, in which a principal chooses incentives to maximize utility while anticipating the agent’s best response. This problem, central to moral hazard and contract theory, underlies applications ranging from market design to delegated portfolio management, hedge fund fee structures, and executive compensation. While linear-quadratic models such as Holmstr"om-Milgrom admit closed-form solutions, realistic environments with nonlinear utilities, stochastic dynamics, or high-dimensional actions generally do not. We introduce a generic algorithmic framework that removes this reliance on closed forms. Our method adapts modern machine learning techniques for bilevel optimization – using implicit differentiation with conjugate gradients (CG) – to compute hypergradients efficiently through Hessian-vector products, without ever forming or inverting Hessians. In benchmark CARA-Normal (Constant Absolute Risk Aversion with Gaussian distribution of uncertainty) environments, the approach recovers known analytical optima and converges reliably from random initialization. More broadly, because it is matrix-free, variance-reduced, and problem-agnostic, the framework extends naturally to complex nonlinear contracts where closed-form solutions are unavailable, such as sigmoidal wage schedules (logistic pay), relative-performance/tournament compensation with common shocks, multi-task contracts with vector actions and heterogeneous noise, and CARA-Poisson count models with $\mathbb{E}[X\mid a]=e^{a}$. This provides a new computational tool for contract design, enabling systematic study of models that have remained analytically intractable.

Method: Extends LoRA by introducing orthogonal basis vectors for each task, using perturbation-based optimization to minimize conflicts, and employing a selection mechanism to identify basis vectors with minimal interference sensitivity.

Result: Empirical results show PLAN consistently outperforms existing methods on standard CL benchmarks, establishing new state-of-the-art performance.

Conclusion: PLAN provides an effective framework for continual learning with foundation models by proactively managing task-specific subspaces to reduce interference while maintaining efficient adaptation.

Abstract: Continual learning (CL) requires models to continuously adapt to new tasks without forgetting past knowledge. In this work, we propose \underline{P}roactive \underline{L}ow-rank \underline{A}llocatio\underline{N} (PLAN), a framework that extends Low-Rank Adaptation (LoRA) to enable efficient and interference-aware fine-tuning of large pre-trained models in CL settings. PLAN proactively manages the allocation of task-specific subspaces by introducing orthogonal basis vectors for each task and optimizing them through a perturbation-based strategy that minimizes conflicts with previously learned parameters. Furthermore, PLAN incorporates a novel selection mechanism that identifies and assigns basis vectors with minimal sensitivity to interference, reducing the risk of degrading past knowledge while maintaining efficient adaptation to new tasks. Empirical results on standard CL benchmarks demonstrate that PLAN consistently outperforms existing methods, establishing a new state-of-the-art for continual learning with foundation models.

Method: Created GenReview dataset by generating 81K reviews for all ICLR 2018-2025 submissions using three independent prompts (negative, positive, neutral) with LLMs, and linked these to original papers and reviews.

Result: LLMs exhibit bias in reviewing, LLM-written reviews can be automatically detected, LLMs don’t always follow reviewing instructions rigorously, and LLM-provided ratings only align with acceptance decisions for accepted papers.

Conclusion: GenReview enables comprehensive investigation of LLM utility and implications in scientific peer review, revealing biases and limitations while providing a valuable resource for future research on AI-assisted reviewing.

Abstract: How does the progressive embracement of Large Language Models (LLMs) affect scientific peer reviewing? This multifaceted question is fundamental to the effectiveness – as well as to the integrity – of the scientific process. Recent evidence suggests that LLMs may have already been tacitly used in peer reviewing, e.g., at the 2024 International Conference of Learning Representations (ICLR). Furthermore, some efforts have been undertaken in an attempt to explicitly integrate LLMs in peer reviewing by various editorial boards (including that of ICLR'25). To fully understand the utility and the implications of LLMs’ deployment for scientific reviewing, a comprehensive relevant dataset is strongly desirable. Despite some previous research on this topic, such dataset has been lacking so far. We fill in this gap by presenting GenReview, the hitherto largest dataset containing LLM-written reviews. Our dataset includes 81K reviews generated for all submissions to the 2018–2025 editions of the ICLR by providing the LLM with three independent prompts: a negative, a positive, and a neutral one. GenReview is also linked to the respective papers and their original reviews, thereby enabling a broad range of investigations. To illustrate the value of GenReview, we explore a sample of intriguing research questions, namely: if LLMs exhibit bias in reviewing (they do); if LLM-written reviews can be automatically detected (so far, they can); if LLMs can rigorously follow reviewing instructions (not always) and whether LLM-provided ratings align with decisions on paper acceptance or rejection (holds true only for accepted papers). GenReview can be accessed at the following link: https://anonymous.4open.science/r/gen_review.

Method: Develops a novel second-order surrogate loss based on pairwise hinge loss, directly substituting entire aggregated pairwise loss with surrogate using first- and second-order statistics. Extends to nonlinear settings via kernel formulation.

Result: Achieves tighter O(ln T) regret bound compared to conventional O(√T) bounds. Extensive experiments show superior efficiency and effectiveness on benchmark datasets.

Conclusion: The proposed second-order surrogate loss framework provides an efficient and effective solution for online AUC optimization with improved theoretical guarantees.

Abstract: The Area Under the Curve (AUC) is an important performance metric for classification tasks, particularly in class-imbalanced scenarios. However, minimizing the AUC presents significant challenges due to the non-convex and discontinuous nature of pairwise 0/1 losses, which are difficult to optimize, as well as the substantial memory cost of instance-wise storage, which creates bottlenecks in large-scale applications. To overcome these challenges, we propose a novel second-order surrogate loss based on the pairwise hinge loss, and develop an efficient online algorithm. Unlike conventional approaches that approximate each individual pairwise 0/1 loss term with an instance-wise surrogate function, our approach introduces a new paradigm that directly substitutes the entire aggregated pairwise loss with a surrogate loss function constructed from the first- and second-order statistics of the training data. Theoretically, while existing online AUC optimization algorithms typically achieve an $\mathcal{O}(\sqrt{T})$ regret bound, our method attains a tighter $\mathcal{O}(\ln T)$ bound. Furthermore, we extend the proposed framework to nonlinear settings through a kernel-based formulation. Extensive experiments on multiple benchmark datasets demonstrate the superior efficiency and effectiveness of the proposed second-order surrogate loss in optimizing online AUC performance.

Method: Systematically infuses counterfactual explanations (CFEs) - inputs that flip teacher predictions with minimal perturbation - to precisely map teacher’s decision boundaries using significantly fewer samples.

Result: CoD outperforms standard distillation approaches in few-shot regimes (8-512 samples), achieving better performance using only half the original samples paired with their CFEs across various datasets and LLMs.

Conclusion: Counterfactual explanations provide theoretical and practical benefits for few-shot knowledge distillation, enabling effective knowledge transfer with minimal data requirements.

Abstract: Knowledge distillation is a promising approach to transfer capabilities from complex teacher models to smaller, resource-efficient student models that can be deployed easily, particularly in task-aware scenarios. However, existing methods of task-aware distillation typically require substantial quantities of data which may be unavailable or expensive to obtain in many practical scenarios. In this paper, we address this challenge by introducing a novel strategy called Counterfactual-explanation-infused Distillation CoD for few-shot task-aware knowledge distillation by systematically infusing counterfactual explanations. Counterfactual explanations (CFEs) refer to inputs that can flip the output prediction of the teacher model with minimum perturbation. Our strategy CoD leverages these CFEs to precisely map the teacher’s decision boundary with significantly fewer samples. We provide theoretical guarantees for motivating the role of CFEs in distillation, from both statistical and geometric perspectives. We mathematically show that CFEs can improve parameter estimation by providing more informative examples near the teacher’s decision boundary. We also derive geometric insights on how CFEs effectively act as knowledge probes, helping the students mimic the teacher’s decision boundaries more effectively than standard data. We perform experiments across various datasets and LLMs to show that CoD outperforms standard distillation approaches in few-shot regimes (as low as 8-512 samples). Notably, CoD only uses half of the original samples used by the baselines, paired with their corresponding CFEs and still improves performance.

Method: Systematically investigate key properties of synthetic priors that allow pretrained TFMs to generalize well, then introduce Mitra - a TFM trained on a curated mixture of synthetic priors selected for diversity, distinctiveness, and performance on real-world data.

Result: Mitra consistently outperforms state-of-the-art TFMs (TabPFNv2 and TabICL) across both classification and regression benchmarks with better sample efficiency.

Conclusion: The work provides the first systematic investigation of synthetic prior design principles for tabular foundation models and demonstrates that carefully curated mixtures of synthetic priors can significantly improve model performance and sample efficiency.

Abstract: Since the seminal work of TabPFN, research on tabular foundation models (TFMs) based on in-context learning (ICL) has challenged long-standing paradigms in machine learning. Without seeing any real-world data, models pretrained on purely synthetic datasets generalize remarkably well across diverse datasets, often using only a moderate number of in-context examples. This shifts the focus in tabular machine learning from model architecture design to the design of synthetic datasets, or, more precisely, to the prior distributions that generate them. Yet the guiding principles for prior design remain poorly understood. This work marks the first attempt to address the gap. We systematically investigate and identify key properties of synthetic priors that allow pretrained TFMs to generalize well. Based on these insights, we introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world tabular data. Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks, with better sample efficiency.

Method: Uses backbone-residual expert architecture with structurally-aware gating network, trained end-to-end with unified unsupervised objective combining reconstruction and diversity regularization.

Result: Achieves state-of-the-art performance in unsupervised node classification and few-shot learning across 16 benchmarks, with superior generalization and training efficiency.

Conclusion: ADaMoRE enables robust, fully unsupervised training of heterogeneous MoE on graphs, improving adaptability to diverse graph structures and tasks.

Abstract: Graph Neural Networks (GNNs) face a fundamental adaptability challenge: their fixed message-passing architectures struggle with the immense diversity of real-world graphs, where optimal computational strategies vary by local structure and task. While Mixture-of-Experts (MoE) offers a promising pathway to adaptability, existing graph MoE methods remain constrained by their reliance on supervised signals and instability when training heterogeneous experts. We introduce ADaMoRE (Adaptive Mixture of Residual Experts), a principled framework that enables robust, fully unsupervised training of heterogeneous MoE on graphs. ADaMoRE employs a backbone-residual expert architecture where foundational encoders provide stability while specialized residual experts capture diverse computational patterns. A structurally-aware gating network performs fine-grained node routing. The entire architecture is trained end-to-end using a unified unsupervised objective, which integrates a primary reconstruction task with an information-theoretic diversity regularizer to explicitly enforce functional specialization among the experts. Theoretical analysis confirms our design improves data efficiency and training stability. Extensive evaluation across 16 benchmarks validates ADaMoRE’s state-of-the-art performance in unsupervised node classification and few-shot learning, alongside superior generalization, training efficiency, and faster convergence on diverse graphs and tasks.

Method: Map flow trajectories to equilibrium states of the 2D Ising model, using an encoder-flow-projector architecture that performs temperature-driven diffusion while preserving physical constraints.

Result: The framework preserves physical fidelity and outperforms Monte Carlo generation in speed for larger lattice sizes, while making each vector field step interpretable as a thermal equilibrium transition.

Conclusion: Embedding physical semantics into generative flows transforms opaque neural trajectories into interpretable physical processes, demonstrating the value of physical constraints in generative modeling.

Abstract: Generative models based on flow matching have demonstrated remarkable success in various domains, yet they suffer from a fundamental limitation: the lack of interpretability in their intermediate generation steps. In fact these models learn to transform noise into data through a series of vector field updates, however the meaning of each step remains opaque. We address this problem by proposing a general framework constraining each flow step to be sampled from a known physical distribution. Flow trajectories are mapped to (and constrained to traverse) the equilibrium states of the simulated physical process. We implement this approach through the 2D Ising model in such a way that flow steps become thermal equilibrium points along a parametric cooling schedule. Our proposed architecture includes an encoder that maps discrete Ising configurations into a continuous latent space, a flow-matching network that performs temperature-driven diffusion, and a projector that returns to discrete Ising states while preserving physical constraints. We validate this framework across multiple lattice sizes, showing that it preserves physical fidelity while outperforming Monte Carlo generation in speed as the lattice size increases. In contrast with standard flow matching, each vector field represents a meaningful stepwise transition in the 2D Ising model’s latent space. This demonstrates that embedding physical semantics into generative flows transforms opaque neural trajectories into interpretable physical processes.

Method: Proposes FDA framework that models input-representation space using synthetic inputs (dual anchors) whose induced gradients align with task vectors, capturing task-specific functional shifts relative to the pretrained model.

Result: FDAs demonstrate effectiveness in model merging and are complementary to parameter-space model merging methods, offering both robustness and flexibility.

Conclusion: FDA framework provides a novel approach to model merging by operating in input-representation space, bridging joint multi-task training and post-hoc merging while being complementary to existing parameter-space methods.

Abstract: Model merging is an efficient post-training strategy for integrating knowledge from multiple finetuned checkpoints of a shared foundation model. Existing methods operate in the parameter space, combining task vectors to mitigate conflicts, but remain constrained by parameter inconsistencies. We propose Functional Dual Anchors (FDAs), a framework that instead models the input-representation space. FDAs are synthetic inputs whose induced gradients align with task vectors, capturing task-specific functional shifts relative to the pretrained model. This perspective bridges joint multi-task training and post-hoc merging, offering both robustness and flexibility. We further introduce a principled initialization scheme and show that FDAs are complementary to parameter-space model merging. Comprehensive experiments demonstrate the effectiveness of FDAs in model merging.

Method: Formalizes the problem as constrained optimization and proposes an adversarial training procedure to find modified models that are statistically close to the reference model while producing divergent performance between optimal and suboptimal policies.

Result: Empirical study shows that the degree of achievable confusion correlates with uncertainty in the approximate model, suggesting potential for theoretically-grounded exploration strategies.

Conclusion: The proposed adversarial training approach successfully constructs most confusing instances for neural network world models, with confusion levels relating to model uncertainty, providing insights for deep model-based RL exploration.

Abstract: In reinforcement learning (RL) theory, the concept of most confusing instances is central to establishing regret lower bounds, that is, the minimal exploration needed to solve a problem. Given a reference model and its optimal policy, a most confusing instance is the statistically closest alternative model that makes a suboptimal policy optimal. While this concept is well-studied in multi-armed bandits and ergodic tabular Markov decision processes, constructing such instances remains an open question in the general case. In this paper, we formalize this problem for neural network world models as a constrained optimization: finding a modified model that is statistically close to the reference one, while producing divergent performance between optimal and suboptimal policies. We propose an adversarial training procedure to solve this problem and conduct an empirical study across world models of varying quality. Our results suggest that the degree of achievable confusion correlates with uncertainty in the approximate model, which may inform theoretically-grounded exploration strategies for deep model-based RL.

Method: Analyze stochastic gradient Langevin dynamics (SGLD) as an Itô SDE approximation of SGD, focusing on multiplicative and state-dependent noise under Hessian regularity conditions.

Result: SGLD yields non-degenerate kernel throughout training with high probability and achieves exponential convergence to empirical risk minimizer in expectation, with established finite-time and finite-width bounds on optimality gap.

Conclusion: Theoretical analysis provides rigorous convergence guarantees for SGLD in lazy training regime, supported by numerical experiments in regression settings.

Abstract: Continuous-time models provide important insights into the training dynamics of optimization algorithms in deep learning. In this work, we establish a non-asymptotic convergence analysis of stochastic gradient Langevin dynamics (SGLD), which is an It^o stochastic differential equation (SDE) approximation of stochastic gradient descent in continuous time, in the lazy training regime. We show that, under regularity conditions on the Hessian of the loss function, SGLD with multiplicative and state-dependent noise (i) yields a non-degenerate kernel throughout the training process with high probability, and (ii) achieves exponential convergence to the empirical risk minimizer in expectation, and we establish finite-time and finite-width bounds on the optimality gap. We corroborate our theoretical findings with numerical examples in the regression setting.

Method: Developed a theoretical framework to analyze convergence of adaptive optimizers under floating-point quantization of gradients, weights, and optimizer states. Derived convergence rates on smooth non-convex objectives under standard stochastic gradient assumptions.

Result: Both Adam and Muon retain convergence rates close to their full-precision counterparts when mantissa length scales logarithmically with iterations. Adam is more sensitive to weights and second-moment quantization, while Muon requires weaker error control and is more robust.

Conclusion: The framework bridges the gap between empirical success and theoretical understanding of low-precision training methods, with numerical experiments validating the theoretical findings.

Abstract: The rapid scaling of large language models (LLMs) has made low-precision training essential for reducing memory, improving efficiency, and enabling larger models and datasets. Existing convergence theories for adaptive optimizers, however, assume all components are exact and neglect hardware-aware quantization, leaving open the question of why low-precision training remains effective. We introduce the first theoretical framework for analyzing the convergence of adaptive optimizers, including Adam and Muon, under floating-point quantization of gradients, weights, and optimizer states (e.g., moment estimates). Within this framework, we derive convergence rates on smooth non-convex objectives under standard stochastic gradient assumptions, explicitly characterizing how quantization errors from different components affect convergence. We show that both algorithms retain rates close to their full-precision counterparts provided mantissa length scales only logarithmically with the number of iterations. Our analysis further reveals that Adam is highly sensitive to weights and second-moment quantization due to its reliance on $\beta_2 \to 1$, while Muon requires weaker error control and is thus potentially more robust. These results narrow the gap between empirical success and theoretical understanding of low-precision training methods. Numerical experiments on synthetic and real-world data corroborate our theory.

Method: Developed three open-source libraries in Julia and Python that centralize numerous recurrent cell implementations and higher-level recurrent architectures, providing a consistent framework for constructing and extending RNN models.

Result: Created torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl libraries that offer built-in mechanisms for customization and experimentation with RNN models.

Conclusion: The libraries provide a standardized platform for testing RNN variants, addressing reproducibility issues and enabling easier exploration of recurrent architectures, all available under MIT license and actively maintained on GitHub.

Abstract: Recurrent neural networks (RNNs) are a cornerstone of sequence modeling across various scientific and industrial applications. Owing to their versatility, numerous RNN variants have been proposed over the past decade, aiming to improve the modeling of long-term dependencies and to address challenges such as vanishing and exploding gradients. However, no central library is available to test these variations, and reimplementing diverse architectures can be time-consuming and error-prone, limiting reproducibility and exploration. Here, we introduce three open-source libraries in Julia and Python that centralize numerous recurrent cell implementations and higher-level recurrent architectures. torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl offer a consistent framework for constructing and extending RNN models, providing built-in mechanisms for customization and experimentation. All packages are available under the MIT license and actively maintained on GitHub.

Method: Uses local Mixture of Experts combining parameter-efficient ensemble models and sparse coding, with Adversarial Adaptive Sampling for fully learnable domain decomposition that adapts to the PDE and its domain.

Result: Achieves better accuracy than state-of-the-art in scientific machine learning while maintaining scalability properties.

Conclusion: PINN Balls successfully enables second-order training of PINNs through efficient architecture design and adaptive domain decomposition, drawing from sound theoretical foundations.

Abstract: Recent advances in Scientific Machine Learning have shown that second-order methods can enhance the training of Physics-Informed Neural Networks (PINNs), making them a suitable alternative to traditional numerical methods for Partial Differential Equations (PDEs). However, second-order methods induce large memory requirements, making them scale poorly with the model size. In this paper, we define a local Mixture of Experts (MoE) combining the parameter-efficiency of ensemble models and sparse coding to enable the use of second-order training. Our model – \textsc{PINN Balls} – also features a fully learnable domain decomposition structure, achieved through the use of Adversarial Adaptive Sampling (AAS), which adapts the DD to the PDE and its domain. \textsc{PINN Balls} achieves better accuracy than the state-of-the-art in scientific machine learning, while maintaining invaluable scalability properties and drawing from a sound theoretical background.

Method: RAVEN dynamically learns optimal combinations of weak models in addition to parameters of the strong model, enabling robust supervision across distribution shifts.

Result: RAVEN outperforms alternative baselines by over 30% on out-of-distribution tasks while matching or surpassing existing methods on in-distribution tasks across image classification, text classification, and preference alignment tasks.

Conclusion: RAVEN effectively addresses distribution shift challenges in weak-to-strong generalization, automatically identifying trustworthy supervision by assigning higher weights to more accurate weak models.

Abstract: As future superhuman models become increasingly complex, accurately supervising their behavior may exceed human capabilities. Recent works have demonstrated that in such scenarios, weak models can effectively supervise strong models, a phenomenon known as weak-to-strong generalization. However, we find that naive weak-to-strong generalization fails under distribution shifts, often leading to worse performance of the strong model than its weak supervisors. To address this, we propose RAVEN, a robust weak-to-strong generalization framework that dynamically learns the optimal combinations of weak models in addition to parameters of the strong model. We demonstrate the effectiveness of RAVEN on image classification, text classification, and preference alignment tasks. RAVEN outperforms alternative baselines by over 30% on out-of-distribution tasks while matching or surpassing existing methods on in-distribution tasks. Moreover, our results show that RAVEN assigns higher weights to more accurate weak models, demonstrating its ability to automatically identify trustworthy supervision.

Method: Wideformer divides aggregation of all nodes into parallel processes to limit input volume per aggregation, and uses a guiding step that sorts and weights aggregation outputs to prioritize informative messages.

Result: Evaluations show Wideformer effectively mitigates over-aggregating, allowing backbone methods to focus on informative messages and achieve superior performance compared to baseline methods.

Conclusion: Wideformer successfully addresses the over-aggregating problem in graph attention through parallel aggregation processes and guided message prioritization, improving model performance.

Abstract: Graph attention has demonstrated superior performance in graph learning tasks. However, learning from global interactions can be challenging due to the large number of nodes. In this paper, we discover a new phenomenon termed over-aggregating. Over-aggregating arises when a large volume of messages is aggregated into a single node with less discrimination, leading to the dilution of the key messages and potential information loss. To address this, we propose Wideformer, a plug-and-play method for graph attention. Wideformer divides the aggregation of all nodes into parallel processes and guides the model to focus on specific subsets of these processes. The division can limit the input volume per aggregation, avoiding message dilution and reducing information loss. The guiding step sorts and weights the aggregation outputs, prioritizing the informative messages. Evaluations show that Wideformer can effectively mitigate over-aggregating. As a result, the backbone methods can focus on the informative messages, achieving superior performance compared to baseline methods.

Method: Introduces a Buffer layer that preserves pre-trained backbone integrity instead of modifying core parameters, preventing catastrophic forgetting during online adaptation.

Result: Outperforms traditional methods in domain shift mitigation and robustness, shows strong resilience to forgetting, and integrates seamlessly into existing TTA frameworks with consistent performance improvements.

Conclusion: The Buffer layer provides an effective and versatile solution for real-world domain adaptation scenarios, overcoming fundamental limitations of normalization-based approaches.

Abstract: In recent advancements in Test Time Adaptation (TTA), most existing methodologies focus on updating normalization layers to adapt to the test domain. However, the reliance on normalization-based adaptation presents key challenges. First, normalization layers such as Batch Normalization (BN) are highly sensitive to small batch sizes, leading to unstable and inaccurate statistics. Moreover, normalization-based adaptation is inherently constrained by the structure of the pre-trained model, as it relies on training-time statistics that may not generalize well to unseen domains. These issues limit the effectiveness of normalization-based TTA approaches, especially under significant domain shift. In this paper, we introduce a novel paradigm based on the concept of a Buffer layer, which addresses the fundamental limitations of normalization layer updates. Unlike existing methods that modify the core parameters of the model, our approach preserves the integrity of the pre-trained backbone, inherently mitigating the risk of catastrophic forgetting during online adaptation. Through comprehensive experimentation, we demonstrate that our approach not only outperforms traditional methods in mitigating domain shift and enhancing model robustness, but also exhibits strong resilience to forgetting. Furthermore, our Buffer layer is modular and can be seamlessly integrated into nearly all existing TTA frameworks, resulting in consistent performance improvements across various architectures. These findings validate the effectiveness and versatility of the proposed solution in real-world domain adaptation scenarios. The code is available at https://github.com/hyeongyu-kim/Buffer_TTA.

Method: Train four independent PatchTST transformer models (one per sensor location) on tampered training data with 1-second sliding windows augmented using jitter, scaling, rotation, and channel dropout to mimic test-time noise. At inference, average softmax probabilities from all four sensor models.

Result: Achieved macro-F1 score substantially above baseline on the private leaderboard of the 2nd WEAR Dataset Challenge.

Conclusion: Test-matched augmentation combined with transformer-based ensembling is an effective strategy for robust human activity recognition under noisy conditions.

Abstract: We present a noise-aware, sensor-specific ensemble approach for robust human activity recognition on the 2nd WEAR Dataset Challenge. Our method leverages the PatchTST transformer architecture, training four independent models-one per inertial sensor location-on a tampered training set whose 1-second sliding windows are augmented to mimic the test-time noise. By aligning the train and test data schemas (JSON-encoded 50-sample windows) and applying randomized jitter, scaling, rotation, and channel dropout, each PatchTST model learns to generalize across real-world sensor perturbations. At inference, we compute softmax probabilities from all four sensor models on the Kaggle test set and average them to produce final labels. On the private leaderboard, this pipeline achieves a macro-F1 substantially above the baseline, demonstrating that test-matched augmentation combined with transformer-based ensembling is an effective strategy for robust HAR under noisy conditions.

Method: Proposes a new class of reparameterization methods for transfer learning, using theoretical analysis with Random Matrix Theory in high-dimensional binary classification and validating with practical experiments including LLM fine-tuning.

Result: The approach demonstrates effectiveness in enhancing generalization ability, with theoretical foundations established and practical validation through experiments.

Conclusion: The new reparameterization methods successfully improve generalization in fine-tuning scenarios, offering a theoretically-grounded alternative to existing approaches like LoRA.

Abstract: Fine-tuning has proven to be highly effective in adapting pre-trained models to perform better on new desired tasks with minimal data samples. Among the most widely used approaches are reparameterization methods, which update a target module by augmenting its frozen weight matrix with an additional trainable weight matrix. The most prominent example is Low Rank Adaption (LoRA), which gained significant attention in recent years. In this paper, we introduce a new class of reparameterization methods for transfer learning, designed to enhance the generalization ability of fine-tuned models. We establish the effectiveness of our approach in a high-dimensional binary classification setting using tools from Random Matrix Theory, and further validate our theoretical findings through more realistic experiments, such as fine-tuning LLMs.

Method: Proposes DVC method that decomposes data contribution into Layer Value Contribution (LVC) and Global Value Contribution (GVC) using six metrics across quality, relevance, and distributional diversity dimensions. Integrates with UCB algorithm for adaptive source selection.

Result: Extensive experiments across six datasets and eight baselines show DVC consistently outperforms existing approaches under various budget constraints, achieving superior accuracy and F1 scores.

Conclusion: DVC represents the first systematic treatment of hierarchical data evaluation for neural networks, providing both theoretical guarantees and practical advantages for large-scale machine learning systems.

Abstract: Data selection is one of the fundamental problems in neural network training, particularly for multi-layer perceptrons (MLPs) where identifying the most valuable training samples from massive, multi-source, and heterogeneous data sources under budget constraints poses significant challenges. Existing data selection methods, including coreset construction, data Shapley values, and influence functions, suffer from critical limitations: they oversimplify nonlinear transformations, ignore informative intermediate representations in hidden layers, or fail to scale to larger MLPs due to high computational complexity. In response, we propose DVC (Data Value Contribution), a novel budget-aware method for evaluating and selecting data for MLP training that accounts for the dynamic evolution of network parameters during training. The DVC method decomposes data contribution into Layer Value Contribution (LVC) and Global Value Contribution (GVC), employing six carefully designed metrics and corresponding efficient algorithms to capture data characteristics across three dimensions–quality, relevance, and distributional diversity–at different granularities. DVC integrates these assessments with an Upper Confidence Bound (UCB) algorithm for adaptive source selection that balances exploration and exploitation. Extensive experiments across six datasets and eight baselines demonstrate that our method consistently outperforms existing approaches under various budget constraints, achieving superior accuracy and F1 scores. Our approach represents the first systematic treatment of hierarchical data evaluation for neural networks, providing both theoretical guarantees and practical advantages for large-scale machine learning systems.

Method: Theoretical complexity analysis under standard assumptions (P!=NP), examining different forms of GAMs, input space structures, component models, and explanation methods across regression and classification contexts.

Result: Found surprising complexity diversity: (1) input space structure heavily influences explanation complexity, (2) component model differences only emerge in specific input domains, (3) significant complexity distinctions between regression and classification tasks, (4) additive representations like neural additive models can make explanation easier but only for certain methods and domains.

Conclusion: The study provides a rigorous theoretical framework showing that computing explanations for GAMs is not universally easy, with feasibility depending on specific conditions, input domains, explanation methods, and task types.

Abstract: Generalized Additive Models (GAMs) are commonly considered interpretable within the ML community, as their structure makes the relationship between inputs and outputs relatively understandable. Therefore, it may seem natural to hypothesize that obtaining meaningful explanations for GAMs could be performed efficiently and would not be computationally infeasible. In this work, we challenge this hypothesis by analyzing the computational complexity of generating different explanations for various forms of GAMs across multiple contexts. Our analysis reveals a surprisingly diverse landscape of both positive and negative complexity outcomes. Particularly, under standard complexity assumptions such as P!=NP, we establish several key findings: (1) in stark contrast to many other common ML models, the complexity of generating explanations for GAMs is heavily influenced by the structure of the input space; (2) the complexity of explaining GAMs varies significantly with the types of component models used - but interestingly, these differences only emerge under specific input domain settings; (3) significant complexity distinctions appear for obtaining explanations in regression tasks versus classification tasks in GAMs; and (4) expressing complex models like neural networks additively (e.g., as neural additive models) can make them easier to explain, though interestingly, this benefit appears only for certain explanation methods and input domains. Collectively, these results shed light on the feasibility of computing diverse explanations for GAMs, offering a rigorous theoretical picture of the conditions under which such computations are possible or provably hard.

Method: User study with 8 sleep medicine practitioners scoring nocturnal arousal events under three conditions: manual scoring, black-box AI assistance, and transparent white-box AI assistance, with assistance provided either at start or as post-hoc quality-control review.

Result: Both AI and human-AI teams significantly outperform unaided experts. Transparent AI assistance yields 30% event-level performance improvement over black-box AI. Quality-control timing enhances count-based outcomes. 7 out of 8 participants prefer transparency and would adopt the system.

Conclusion: Strategically timed transparent AI assistance effectively balances accuracy and clinical efficiency, providing a promising pathway for trustworthy AI integration and user acceptance in clinical workflows.

Abstract: Artificial intelligence (AI) systems increasingly match or surpass human experts in biomedical signal interpretation. However, their effective integration into clinical practice requires more than high predictive accuracy. Clinicians must discern \textit{when} and \textit{why} to trust algorithmic recommendations. This work presents an application-grounded user study with eight professional sleep medicine practitioners, who score nocturnal arousal events in polysomnographic data under three conditions: (i) manual scoring, (ii) black-box (BB) AI assistance, and (iii) transparent white-box (WB) AI assistance. Assistance is provided either from the \textit{start} of scoring or as a post-hoc quality-control (\textit{QC}) review. We systematically evaluate how the type and timing of assistance influence event-level and clinically most relevant count-based performance, time requirements, and user experience. When evaluated against the clinical standard used to train the AI, both AI and human-AI teams significantly outperform unaided experts, with collaboration also reducing inter-rater variability. Notably, transparent AI assistance applied as a targeted QC step yields median event-level performance improvements of approximately 30% over black-box assistance, and QC timing further enhances count-based outcomes. While WB and QC approaches increase the time required for scoring, start-time assistance is faster and preferred by most participants. Participants overwhelmingly favor transparency, with seven out of eight expressing willingness to adopt the system with minor or no modifications. In summary, strategically timed transparent AI assistance effectively balances accuracy and clinical efficiency, providing a promising pathway toward trustworthy AI integration and user acceptance in clinical workflows.

Method: EPHAD updates AD model outputs at test time by integrating prior knowledge from contaminated training with evidence from multimodal foundation models (like CLIP), classical AD methods, or domain-specific knowledge.

Result: Comprehensive experiments across 8 visual AD datasets, 26 tabular AD datasets, and a real-world industrial dataset show effectiveness, with ablation studies demonstrating robustness to varying contamination levels.

Conclusion: EPHAD provides a versatile and robust framework that improves AD performance on contaminated datasets without requiring training pipeline access or prior knowledge of anomaly proportions.

Abstract: Unsupervised anomaly detection (AD) methods typically assume clean training data, yet real-world datasets often contain undetected or mislabeled anomalies, leading to significant performance degradation. Existing solutions require access to the training pipelines, data or prior knowledge of the proportions of anomalies in the data, limiting their real-world applicability. To address this challenge, we propose EPHAD, a simple yet effective test-time adaptation framework that updates the outputs of AD models trained on contaminated datasets using evidence gathered at test time. Our approach integrates the prior knowledge captured by the AD model trained on contaminated datasets with evidence derived from multimodal foundation models like Contrastive Language-Image Pre-training (CLIP), classical AD methods like the Latent Outlier Factor or domain-specific knowledge. We illustrate the intuition behind EPHAD using a synthetic toy example and validate its effectiveness through comprehensive experiments across eight visual AD datasets, twenty-six tabular AD datasets, and a real-world industrial AD dataset. Additionally, we conduct an ablation study to analyse hyperparameter influence and robustness to varying contamination levels, demonstrating the versatility and robustness of EPHAD across diverse AD models and evidence pairs. To ensure reproducibility, our code is publicly available at https://github.com/sukanyapatra1997/EPHAD.

Method: Uses amortized variational inference with neural networks to predict variational parameters from input data, directly approximating the posterior distribution over true labels. This combines deep learning expressiveness with probabilistic modeling rigor while remaining architecture-agnostic.

Result: Theoretical analysis and extensive experiments on synthetic and real-world datasets demonstrate state-of-the-art performance in both accuracy and efficiency.

Conclusion: The proposed probabilistic framework effectively addresses partial-label learning challenges by combining the strengths of deep learning and probabilistic modeling, providing an efficient and accurate solution for noisy, ambiguous data scenarios.

Abstract: Real-world data is frequently noisy and ambiguous. In crowdsourcing, for example, human annotators may assign conflicting class labels to the same instances. Partial-label learning (PLL) addresses this challenge by training classifiers when each instance is associated with a set of candidate labels, only one of which is correct. While early PLL methods approximate the true label posterior, they are often computationally intensive. Recent deep learning approaches improve scalability but rely on surrogate losses and heuristic label refinement. We introduce a novel probabilistic framework that directly approximates the posterior distribution over true labels using amortized variational inference. Our method employs neural networks to predict variational parameters from input data, enabling efficient inference. This approach combines the expressiveness of deep learning with the rigor of probabilistic modeling, while remaining architecture-agnostic. Theoretical analysis and extensive experiments on synthetic and real-world datasets demonstrate that our method achieves state-of-the-art performance in both accuracy and efficiency.

Method: Adapted cognitive neuroscience associative learning paradigm using LLMs’ in-context learning across six models, analyzing representational evolution and vocabulary interference effects.

Result: Found non-monotonic plasticity pattern with moderate item differentiation after learning, amplified by higher vocabulary interference between new associations and prior knowledge.

Conclusion: LLMs serve as powerful tools for studying representational dynamics and generating hypotheses about memory reorganization principles in biological systems.

Abstract: Associative learning–forming links between co-occurring items–is fundamental to human cognition, reshaping internal representations in complex ways. Testing hypotheses on how representational changes occur in biological systems is challenging, but large language models (LLMs) offer a scalable alternative. Building on LLMs’ in-context learning, we adapt a cognitive neuroscience associative learning paradigm and investigate how representations evolve across six models. Our initial findings reveal a non-monotonic pattern consistent with the Non-Monotonic Plasticity Hypothesis, with moderately similar items differentiating after learning. Leveraging the controllability of LLMs, we further show that this differentiation is modulated by the overlap of associated items with the broader vocabulary–a factor we term vocabulary interference, capturing how new associations compete with prior knowledge. We find that higher vocabulary interference amplifies differentiation, suggesting that representational change is influenced by both item similarity and global competition. Our findings position LLMs not only as powerful tools for studying representational dynamics in human-like learning systems, but also as accessible and general computational models for generating new hypotheses about the principles underlying memory reorganization in the brain.

Method: Introduces a neighbouring datasets framework to study single-data-point effects on multiplicity. Extends this to active learning and data imputation with novel multiplicity-aware algorithms.

Result: Counterintuitive finding: neighbouring datasets with greater inter-class distribution overlap exhibit lower multiplicity, explained by a shared Rashomon parameter. Provides rigorous proofs and applies framework to practical domains.

Conclusion: Data characteristics significantly influence multiplicity, with inter-class overlap reducing it. The framework enables systematic study and development of multiplicity-aware methods for active learning and data imputation.

Abstract: Multiplicity – the existence of distinct models with comparable performance – has received growing attention in recent years. While prior work has largely emphasized modelling choices, the critical role of data in shaping multiplicity has been comparatively overlooked. In this work, we introduce a neighbouring datasets framework to examine the most granular case: the impact of a single-data-point difference on multiplicity. Our analysis yields a seemingly counterintuitive finding: neighbouring datasets with greater inter-class distribution overlap exhibit lower multiplicity. This reversal of conventional expectations arises from a shared Rashomon parameter, and we substantiate it with rigorous proofs. Building on this foundation, we extend our framework to two practical domains: active learning and data imputation. For each, we establish natural extensions of the neighbouring datasets perspective, conduct the first systematic study of multiplicity in existing algorithms, and finally, propose novel multiplicity-aware methods, namely, multiplicity-aware data acquisition strategies for active learning and multiplicity-aware data imputation techniques.

Method: Adds prioritized replay buffer scoring trajectories by return, reconstruction loss, and value error, plus intrinsic reward based on ensemble disagreement over predicted rewards.

Result: Faster learning and lower dynamics model loss on Atari100k and DeepMind Control Visual Benchmark tasks, especially in sparse-reward settings.

Conclusion: The proposed extensions successfully improve DreamerV3’s performance, confirming the original results while achieving better exploration and efficiency.

Abstract: We introduce DreamerV3-XP, an extension of DreamerV3 that improves exploration and learning efficiency. This includes (i) a prioritized replay buffer, scoring trajectories by return, reconstruction loss, and value error and (ii) an intrinsic reward based on disagreement over predicted environment rewards from an ensemble of world models. DreamerV3-XP is evaluated on a subset of Atari100k and DeepMind Control Visual Benchmark tasks, confirming the original DreamerV3 results and showing that our extensions lead to faster learning and lower dynamics model loss, particularly in sparse-reward settings.

Method: Use initial uniform exploration phase followed by standard Upper Confidence Bound (UCB) algorithm, relying only on additive Hoeffding bounds and extending to sub-Gaussian rewards.

Result: Achieves near-optimal Nash regret bounds, generalizes to p-mean regret metrics, and works uniformly across all p values with fewer assumptions than prior work.

Conclusion: Simple algorithms with uniform exploration can achieve optimal fairness-aware regret bounds without restrictive assumptions, making fair bandit algorithms more practical and broadly applicable.

Abstract: Regret in stochastic multi-armed bandits traditionally measures the difference between the highest reward and either the arithmetic mean of accumulated rewards or the final reward. These conventional metrics often fail to address fairness among agents receiving rewards, particularly in settings where rewards are distributed across a population, such as patients in clinical trials. To address this, a recent body of work has introduced Nash regret, which evaluates performance via the geometric mean of accumulated rewards, aligning with the Nash social welfare function known for satisfying fairness axioms. To minimize Nash regret, existing approaches require specialized algorithm designs and strong assumptions, such as multiplicative concentration inequalities and bounded, non-negative rewards, making them unsuitable for even Gaussian reward distributions. We demonstrate that an initial uniform exploration phase followed by a standard Upper Confidence Bound (UCB) algorithm achieves near-optimal Nash regret, while relying only on additive Hoeffding bounds, and naturally extending to sub-Gaussian rewards. Furthermore, we generalize the algorithm to a broad class of fairness metrics called the $p$-mean regret, proving (nearly) optimal regret bounds uniformly across all $p$ values. This is in contrast to prior work, which made extremely restrictive assumptions on the bandit instances and even then achieved suboptimal regret bounds.

Method: Proposed ScoreNF framework combines score-based learning with Normalizing Flow architecture integrated with Independent Metropolis-Hastings module for sampling from unnormalized distributions.

Result: ScoreNF maintains high performance with small training ensembles, reduces reliance on expensive MCMC-generated data, and includes method for assessing mode-covering/collapse behaviors. Validated on synthetic 2D distributions and high-dimensional φ⁴ lattice field theory.

Conclusion: ScoreNF provides an effective framework for sampling from unnormalized distributions with improved efficiency and reduced data requirements compared to traditional methods.

Abstract: Unnormalized probability distributions are central to modeling complex physical systems across various scientific domains. Traditional sampling methods, such as Markov Chain Monte Carlo (MCMC), often suffer from slow convergence, critical slowing down, poor mode mixing, and high autocorrelation. In contrast, likelihood-based and adversarial machine learning models, though effective, are heavily data-driven, requiring large datasets and often encountering mode covering and mode collapse. In this work, we propose ScoreNF, a score-based learning framework built on the Normalizing Flow (NF) architecture, integrated with an Independent Metropolis-Hastings (IMH) module, enabling efficient and unbiased sampling from unnormalized target distributions. We show that ScoreNF maintains high performance even with small training ensembles, thereby reducing reliance on computationally expensive MCMC-generated training data. We also present a method for assessing mode-covering and mode-collapse behaviours. We validate our method on synthetic 2D distributions (MOG-4 and MOG-8) and the high-dimensional $\phi^4$ lattice field theory distribution, demonstrating its effectiveness for sampling tasks.

Method: Neural network-based framework that estimates yield curves independently per day with a novel loss function enforcing smoothness and stability, plus integration of domain-specific constraints like risk-free benchmark alignment.

Result: Empirical tests on Swedish mortgage bonds show the approach provides more robust and stable yield curve estimates compared to Nelson-Siegel-Svensson and Kernel-Ridge methods.

Conclusion: The framework enables practitioners to balance smoothness and accuracy trade-offs while handling limited and noisy data in small bond markets.

Abstract: Robust yield curve estimation is crucial in fixed-income markets for accurate instrument pricing, effective risk management, and informed trading strategies. Traditional approaches, including the bootstrapping method and parametric Nelson-Siegel models, often struggle with overfitting or instability issues, especially when underlying bonds are sparse, bond prices are volatile, or contain hard-to-remove noise. In this paper, we propose a neural networkbased framework for robust yield curve estimation tailored to small mortgage bond markets. Our model estimates the yield curve independently for each day and introduces a new loss function to enforce smoothness and stability, addressing challenges associated with limited and noisy data. Empirical results on Swedish mortgage bonds demonstrate that our approach delivers more robust and stable yield curve estimates compared to existing methods such as Nelson-Siegel-Svensson (NSS) and Kernel-Ridge (KR). Furthermore, the framework allows for the integration of domain-specific constraints, such as alignment with risk-free benchmarks, enabling practitioners to balance the trade-off between smoothness and accuracy according to their needs.

Method: C-MCTD introduces three components: (1) Online Composer for globally-aware planning across entire plan compositions, (2) Distributed Composer for parallel exploration from multiple starting points to reduce search complexity, and (3) Preplan Composer that leverages cached plan graphs to accelerate inference.

Result: The framework elevates planning from individual trajectory optimization to reasoning over complete plan compositions, enabling longer plan generation and more effective trajectory exploration.

Conclusion: C-MCTD provides a compositional approach to Monte Carlo Tree Diffusion that overcomes the fundamental limitations of MCTD by enabling global reasoning and efficient plan composition.

Abstract: Monte Carlo Tree Diffusion (MCTD) integrates diffusion models with structured tree search to enable effective trajectory exploration through stepwise reasoning. However, MCTD remains fundamentally limited by training trajectory lengths. While periodic replanning allows plan concatenation for longer plan generation, the planning process remains locally confined, as MCTD searches within individual trajectories without access to global context. We propose Compositional Monte Carlo Tree Diffusion (C-MCTD), a framework that elevates planning from individual trajectory optimization to reasoning over complete plan compositions. C-MCTD introduces three complementary components: (1) Online Composer, which performs globally-aware planning by searching across entire plan compositions; (2) Distributed Composer, which reduces search complexity through parallel exploration from multiple starting points; and (3) Preplan Composer, which accelerates inference by leveraging cached plan graphs.

Method: Uses Cholesky decomposition of a target correlation matrix to generate datasets that mathematically preserve the entire correlation structure from pairwise to higher-order interactions.

Result: The method produces synthetic data that preserves the complete correlation structure of the original dataset by mathematical proof.

Conclusion: GCM provides a new approach to synthetic data generation with applications in privacy-preserving data sharing, robust model training, and simulation.

Abstract: The increasing need for data privacy and the demand for robust machine learning models have fueled the development of synthetic data generation techniques. However, current methods often succeed in replicating simple summary statistics but fail to preserve both the pairwise and higher-order correlation structure of the data that define the complex, multi-variable interactions inherent in real-world systems. This limitation can lead to synthetic data that is superficially realistic but fails when used for sophisticated modeling tasks. In this white paper, we introduce Generative Correlation Manifolds (GCM), a computationally efficient method for generating synthetic data. The technique uses Cholesky decomposition of a target correlation matrix to produce datasets that, by mathematical proof, preserve the entire correlation structure – from simple pairwise relationships to higher-order interactions – of the source dataset. We argue that this method provides a new approach to synthetic data generation with potential applications in privacy-preserving data sharing, robust model training, and simulation.

Method: Randomized neural network with forward regularization framework; ensemble deep random vector functional link network (edRVFL) with adjustable forward regularization (-kF); improved to edRVFL-kF-Bayes using Bayesian learning for self-adaptive regularization parameter tuning.

Result: The framework avoids past replay and catastrophic forgetting, generates one-pass closed-form incremental updates with variable learning rates, and achieves superior performance on image datasets across 6 metrics with dynamic performance validation.

Conclusion: The proposed OTCIL frameworks with -kF-Bayes and -kF styles effectively address online task-free CIL challenges, outperforming canonical ridge regularization and demonstrating efficacy through comprehensive experiments.

Abstract: Class incremental learning (CIL) requires an agent to learn distinct tasks consecutively with knowledge retention against forgetting. Problems impeding the practical applications of CIL methods are twofold: (1) non-i.i.d batch streams and no boundary prompts to update, known as the harsher online task-free CIL (OTCIL) scenario; (2) CIL methods suffer from memory loss in learning long task streams, as shown in Fig. 1 (a). To achieve efficient decision-making and decrease cumulative regrets during the OTCIL process, a randomized neural network (Randomized NN) with forward regularization (-F) is proposed to resist forgetting and enhance learning performance. This general framework integrates unsupervised knowledge into recursive convex optimization, has no learning dissipation, and can outperform the canonical ridge style (-R) in OTCIL. Based on this framework, we derive the algorithm of the ensemble deep random vector functional link network (edRVFL) with adjustable forward regularization (-kF), where k mediates the intensity of the intervention. edRVFL-kF generates one-pass closed-form incremental updates and variable learning rates, effectively avoiding past replay and catastrophic forgetting while achieving superior performance. Moreover, to curb unstable penalties caused by non-i.i.d and mitigate intractable tuning of -kF in OTCIL, we improve it to the plug-and-play edRVFL-kF-Bayes, enabling all hard ks in multiple sub-learners to be self-adaptively determined based on Bayesian learning. Experiments were conducted on 2 image datasets including 6 metrics, dynamic performance, ablation tests, and compatibility, which distinctly validates the efficacy of our OTCIL frameworks with -kF-Bayes and -kF styles.

Method: DEEDEE uses a two-statistic approach: episodewise mean and RBF kernel similarity to training summary, capturing both global and local deviations in RL time series.

Result: DEEDEE matches or surpasses contemporary detectors across standard RL OOD suites, with 600x reduction in compute and 5% absolute accuracy gain over baselines.

Conclusion: Diverse anomaly types in RL trajectories can be detected through a small set of low-order statistics, suggesting a compact foundation for OOD detection.

Abstract: Deploying reinforcement learning (RL) in safety-critical settings is constrained by brittleness under distribution shift. We study out-of-distribution (OOD) detection for RL time series and introduce DEEDEE, a two-statistic detector that revisits representation-heavy pipelines with a minimal alternative. DEEDEE uses only an episodewise mean and an RBF kernel similarity to a training summary, capturing complementary global and local deviations. Despite its simplicity, DEEDEE matches or surpasses contemporary detectors across standard RL OOD suites, delivering a 600-fold reduction in compute (FLOPs / wall-time) and an average 5% absolute accuracy gain over strong baselines. Conceptually, our results indicate that diverse anomaly types often imprint on RL trajectories through a small set of low-order statistics, suggesting a compact foundation for OOD detection in complex environments.

Method: Introduces utility functions in freeze-thaw BO framework, novel acquisition functions, dynamic stopping criteria, and transfer learning for improved sample efficiency in cost-sensitive HPO.

Result: Outperforms previous freeze-thaw BO and transfer-BO baselines on established multi-fidelity HPO benchmarks, achieving significantly better cost-performance trade-offs.

Conclusion: The proposed cost-sensitive freeze-thaw BO method effectively balances computational cost and model performance, providing automated stopping around maximum utility with improved efficiency.

Abstract: In this paper, we address the problem of \emph{cost-sensitive} hyperparameter optimization (HPO) built upon freeze-thaw Bayesian optimization (BO). Specifically, we assume a scenario where users want to early-stop the HPO process when the expected performance improvement is not satisfactory with respect to the additional computational cost. Motivated by this scenario, we introduce \emph{utility} in the freeze-thaw framework, a function describing the trade-off between the cost and performance that can be estimated from the user’s preference data. This utility function, combined with our novel acquisition function and stopping criterion, allows us to dynamically continue training the configuration that we expect to maximally improve the utility in the future, and also automatically stop the HPO process around the maximum utility. Further, we improve the sample efficiency of existing freeze-thaw methods with transfer learning to develop a specialized surrogate model for the cost-sensitive HPO problem. We validate our algorithm on established multi-fidelity HPO benchmarks and show that it outperforms all the previous freeze-thaw BO and transfer-BO baselines we consider, while achieving a significantly better trade-off between the cost and performance. Our code is publicly available at https://github.com/db-Lee/CFBO.

Method: Randomly remix latents according to factor-specific mixing strategies (mutually exclusive for attributes, common support for objects) and use two objectives: prior loss for realistic remix decoding and compositional consistency loss for alignment between composite images and latents.

Result: Competitive performance in both attribute and object disentanglement, and uniquely achieves joint disentanglement of global style and objects.

Conclusion: The compositional bias framework provides a flexible approach to disentangled representation learning that can handle various factor types through simple mixing strategy adjustments, eliminating the need for architecture/objective redesigns.

Abstract: Recent disentangled representation learning (DRL) methods heavily rely on factor specific strategies-either learning objectives for attributes or model architectures for objects-to embed inductive biases. Such divergent approaches result in significant overhead when novel factors of variation do not align with prior assumptions, such as statistical independence or spatial exclusivity, or when multiple factors coexist, as practitioners must redesign architectures or objectives. To address this, we propose a compositional bias, a modular inductive bias decoupled from both objectives and architectures. Our key insight is that different factors obey distinct recombination rules in the data distribution: global attributes are mutually exclusive, e.g., a face has one nose, while objects share a common support (any subset of objects can co-exist). We therefore randomly remix latents according to factor-specific rules, i.e., a mixing strategy, and force the encoder to discover whichever factor structure the mixing strategy reflects through two complementary objectives: (i) a prior loss that ensures every remix decodes into a realistic image, and (ii) the compositional consistency loss introduced by Wiedemer et al. (arXiv:2310.05327), which aligns each composite image with its corresponding composite latent. Under this general framework, simply adjusting the mixing strategy enables disentanglement of attributes, objects, and even both, without modifying the objectives or architectures. Extensive experiments demonstrate that our method shows competitive performance in both attribute and object disentanglement, and uniquely achieves joint disentanglement of global style and objects. Code is available at https://github.com/whieya/Compositional-DRL.

Method: Iterative process alternating between noising and denoising steps using a randomly initialized convolutional network trained via data fidelity loss, exploiting neural network spectral bias through scheduled noise.

Result: Achieves competitive or superior performance on various linear inverse problems compared to other methods, without relying on pretrained score functions.

Conclusion: Self-diffusion provides an effective alternative to pretrained diffusion models for inverse problems, offering flexibility and broad applicability across different forward operators.

Abstract: We propose self-diffusion, a novel framework for solving inverse problems without relying on pretrained generative models. Traditional diffusion-based approaches require training a model on a clean dataset to learn to reverse the forward noising process. This model is then used to sample clean solutions – corresponding to posterior sampling from a Bayesian perspective – that are consistent with the observed data under a specific task. In contrast, self-diffusion introduces a self-contained iterative process that alternates between noising and denoising steps to progressively refine its estimate of the solution. At each step of self-diffusion, noise is added to the current estimate, and a self-denoiser, which is a single untrained convolutional network randomly initialized from scratch, is continuously trained for certain iterations via a data fidelity loss to predict the solution from the noisy estimate. Essentially, self-diffusion exploits the spectral bias of neural networks and modulates it through a scheduled noise process. Without relying on pretrained score functions or external denoisers, this approach still remains adaptive to arbitrary forward operators and noisy observations, making it highly flexible and broadly applicable. We demonstrate the effectiveness of our approach on a variety of linear inverse problems, showing that self-diffusion achieves competitive or superior performance compared to other methods.

Method: Replace deep neural networks with extreme learning machine networks, fix input weights, determine output weights by optimizing physical law loss vector, and transform problem into finding Moore-Penrose generalized inverse via least squares.

Result: PIELM increases prediction accuracy by 3-7 orders of magnitude in relative L2 error and saves over 94% training time compared to conventional PINNs.

Conclusion: PIELM provides an efficient and accurate framework for solving inverse Stefan problems, overcoming limitations of PINNs.

Abstract: The inverse Stefan problem, as a typical phase-change problem with moving boundaries, finds extensive applications in science and engineering. Recent years have seen the applications of physics-informed neural networks (PINNs) to solving Stefan problems, yet they still exhibit shortcomings in hyperparameter dependency, training efficiency, and prediction accuracy. To address this, this paper develops a physics-informed extreme learning machine (PIELM), a rapid physics-informed learning method framework for inverse Stefan problems. PIELM replaces conventional deep neural networks with an extreme learning machine network. The input weights are fixed in the PIELM framework, and the output weights are determined by optimizing a loss vector of physical laws composed by initial and boundary conditions and governing partial differential equations (PDEs). Then, solving inverse Stefan problems is transformed into finding the Moore-Penrose generalized inverse by the least squares method. Case studies show that the PIELM can increase the prediction accuracy by 3-7 order of magnitude in terms of the relative L2 error, and meanwhile saving more than 94% training time, compared to conventional PINNs.

Method: Learns sparse local causal masks to identify minimal neighborhood variables, creates approximately compact representations, and uses meta actor-critic with shared policy across domains.

Result: GSAC adapts rapidly with few trajectories, significantly outperforms learning-from-scratch and conventional adaptation baselines.

Conclusion: The framework provides finite-sample guarantees for causal recovery and adaptation, enabling efficient learning on graphs with domain generalization.

Abstract: Large-scale networked systems, such as traffic, power, and wireless grids, challenge reinforcement-learning agents with both scale and environment shifts. To address these challenges, we propose GSAC (Generalizable and Scalable Actor-Critic), a framework that couples causal representation learning with meta actor-critic learning to achieve both scalability and domain generalization. Each agent first learns a sparse local causal mask that provably identifies the minimal neighborhood variables influencing its dynamics, yielding exponentially tight approximately compact representations (ACRs) of state and domain factors. These ACRs bound the error of truncating value functions to $\kappa$-hop neighborhoods, enabling efficient learning on graphs. A meta actor-critic then trains a shared policy across multiple source domains while conditioning on the compact domain factors; at test time, a few trajectories suffice to estimate the new domain factor and deploy the adapted policy. We establish finite-sample guarantees on causal recovery, actor-critic convergence, and adaptation gap, and show that GSAC adapts rapidly and significantly outperforms learning-from-scratch and conventional adaptation baselines.

Method: Developed UTR that unifies return-to-go, state, and action into single tokens. Created two variants: UDT (transformer backbone) and UDC (gated CNN backbone). Theoretical analysis shows tighter Rademacher complexity bound.

Result: Both UDT and UDC achieve comparable or superior performance to state-of-the-art methods with significantly lower computational requirements.

Conclusion: UTR generalizes well across architectures and provides an efficient foundation for scalable control in future large decision models.

Abstract: Transformers have demonstrated strong potential in offline reinforcement learning (RL) by modeling trajectories as sequences of return-to-go, states, and actions. However, existing approaches such as the Decision Transformer(DT) and its variants suffer from redundant tokenization and quadratic attention complexity, limiting their scalability in real-time or resource-constrained settings. To address this, we propose a Unified Token Representation (UTR) that merges return-to-go, state, and action into a single token, substantially reducing sequence length and model complexity. Theoretical analysis shows that UTR leads to a tighter Rademacher complexity bound, suggesting improved generalization. We further develop two variants: UDT and UDC, built upon transformer and gated CNN backbones, respectively. Both achieve comparable or superior performance to state-of-the-art methods with markedly lower computation. These findings demonstrate that UTR generalizes well across architectures and may provide an efficient foundation for scalable control in future large decision models.

Method: Cast nonlinear recurrence relationships as a single system of equations and solve in parallel using Newton’s iterations combined with custom parallel reductions.

Result: Achieved 665x speedup over sequential training, successfully trained 7B parameter LSTM/GRU models with perplexity comparable to similarly-sized Transformers and Mamba2 architectures.

Conclusion: ParaRNN breaks the sequence-parallelization barrier for nonlinear RNNs, enabling scalable training of complex nonlinear sequence models and released as open-source framework.

Abstract: Recurrent Neural Networks (RNNs) laid the foundation for sequence modeling, but their intrinsic sequential nature restricts parallel computation, creating a fundamental barrier to scaling. This has led to the dominance of parallelizable architectures like Transformers and, more recently, State Space Models (SSMs). While SSMs achieve efficient parallelization through structured linear recurrences, this linearity constraint limits their expressive power and precludes modeling complex, nonlinear sequence-wise dependencies. To address this, we present ParaRNN, a framework that breaks the sequence-parallelization barrier for nonlinear RNNs. Building on prior work, we cast the sequence of nonlinear recurrence relationships as a single system of equations, which we solve in parallel using Newton’s iterations combined with custom parallel reductions. Our implementation achieves speedups of up to 665x over naive sequential application, allowing training nonlinear RNNs at unprecedented scales. To showcase this, we apply ParaRNN to adaptations of LSTM and GRU architectures, successfully training models of 7B parameters that attain perplexity comparable to similarly-sized Transformers and Mamba2 architectures. To accelerate research in efficient sequence modeling, we release the ParaRNN codebase as an open-source framework for automatic training-parallelization of nonlinear RNNs, enabling researchers and practitioners to explore new nonlinear RNN models at scale.

Method: Developed a formal framework to estimate authorship probability for user-photo pairs. Used TripAdvisor data containing restaurant reviews with photos from six cities of different sizes to illustrate the approach.

Result: The method can predict attractive images for users and estimate their distribution, helping companies understand aspects that customers highlight about their products.

Conclusion: Personalized explanations can be learned as recommendations themselves through photo prediction, which increases recommender system reliability and provides companies with valuable insights about customer perspectives on their products.

Abstract: Explaining the output of a complex system, such as a Recommender System (RS), is becoming of utmost importance for both users and companies. In this paper we explore the idea that personalized explanations can be learned as recommendation themselves. There are plenty of online services where users can upload some photos, in addition to rating items. We assume that users take these photos to reinforce or justify their opinions about the items. For this reason we try to predict what photo a user would take of an item, because that image is the argument that can best convince her of the qualities of the item. In this sense, an RS can explain its results and, therefore, increase its reliability. Furthermore, once we have a model to predict attractive images for users, we can estimate their distribution. Thus, the companies acquire a vivid knowledge about the aspects that the clients highlight of their products. The paper includes a formal framework that estimates the authorship probability for a given pair (user, photo). To illustrate the proposal, we use data gathered from TripAdvisor containing the reviews (with photos) of restaurants in six cities of different sizes.

Method: Combines graph neural networks with domain adversarial training to handle unknown exposure mappings and mitigate covariate shift effects.

Result: Extensive evaluation on synthetic and semi-synthetic datasets demonstrates the method’s effectiveness.

Conclusion: HINet successfully addresses key challenges in network treatment effect estimation under realistic conditions.

Abstract: Estimating heterogeneous treatment effects in network settings is complicated by interference, meaning that the outcome of an instance can be influenced by the treatment status of others. Existing causal machine learning approaches usually assume a known exposure mapping that summarizes how the outcome of a given instance is influenced by others’ treatment, a simplification that is often unrealistic. Furthermore, the interaction between homophily – the tendency of similar instances to connect – and the treatment assignment mechanism can induce a network-level covariate shift that may lead to inaccurate treatment effect estimates, a phenomenon that has not yet been explicitly studied. To address these challenges, we propose HINet, a novel method that integrates graph neural networks with domain adversarial training. This combination allows estimating treatment effects under unknown exposure mappings while mitigating the impact of (network-level) covariate shift. An extensive empirical evaluation on synthetic and semi-synthetic network datasets demonstrates the effectiveness of our approach.

Method: Proposes a fully linear, parameter-free model with a closed-form solution for weight matrix and redundancy-aware propagation scheme to avoid iterative training and eliminate redundant self-information.

Result: Outperforms 8 baseline models on 11 real-world hypergraph benchmarks with up to 696x speedups over the fastest competitor.

Conclusion: ZEN provides an effective, efficient, and interpretable solution for few-shot hypergraph node classification that avoids the limitations of traditional HNNs.

Abstract: Few-shot node classification on hypergraphs requires models that generalize from scarce labels while capturing high-order structures. Existing hypergraph neural networks (HNNs) effectively encode such structures but often suffer from overfitting and scalability issues due to complex, black-box architectures. In this work, we propose ZEN (Zero-Parameter Hypergraph Neural Network), a fully linear and parameter-free model that achieves both expressiveness and efficiency. Built upon a unified formulation of linearized HNNs, ZEN introduces a tractable closed-form solution for the weight matrix and a redundancy-aware propagation scheme to avoid iterative training and to eliminate redundant self information. On 11 real-world hypergraph benchmarks, ZEN consistently outperforms eight baseline models in classification accuracy while achieving up to 696x speedups over the fastest competitor. Moreover, the decision process of ZEN is fully interpretable, providing insights into the characteristic of a dataset. Our code and datasets are fully available at https://github.com/chaewoonbae/ZEN.

Method: Created a benchmarking framework using Beijing Multi-site Air Quality dataset across 12 decentralized clients. Systematically evaluated CF mitigation strategies including Replay, Elastic Weight Consolidation, Learning without Forgetting, and Synaptic Intelligence.

Result: Provides the first comprehensive comparative analysis of state-of-the-art CF mitigation methods in federated continual time series forecasting setting.

Conclusion: This work delivers essential tools and insights for advancing continual learning in federated time-series forecasting systems, with a reproducible open-source framework released for the community.

Abstract: Catastrophic forgetting (CF) poses a persistent challenge in continual learning (CL), especially within federated learning (FL) environments characterized by non-i.i.d. time series data. While existing research has largely focused on classification tasks in vision domains, the regression-based forecasting setting prevalent in IoT and edge applications remains underexplored. In this paper, we present the first benchmarking framework tailored to investigate CF in federated continual time series forecasting. Using the Beijing Multi-site Air Quality dataset across 12 decentralized clients, we systematically evaluate several CF mitigation strategies, including Replay, Elastic Weight Consolidation, Learning without Forgetting, and Synaptic Intelligence. Key contributions include: (i) introducing a new benchmark for CF in time series FL, (ii) conducting a comprehensive comparative analysis of state-of-the-art methods, and (iii) releasing a reproducible open-source framework. This work provides essential tools and insights for advancing continual learning in federated time-series forecasting systems.

Method: Work on the space of mean vectors of distributions, analyze separability conditions, and construct counterexamples using different techniques to show separability is not necessary.

Result: Separability of mean vectors is sufficient for UME-learnability but not necessary, and countable unions of UME-learnable collections are also UME-learnable.

Conclusion: The paper establishes a more general framework for uniform mean estimation, resolves a conjecture about countable unions, and shows separability is sufficient but not necessary for UME-learnability.

Abstract: We characterize conditions under which collections of distributions on ${0,1}^\mathbb{N}$ admit uniform estimation of their mean. Prior work from Vapnik and Chervonenkis (1971) has focused on uniform convergence using the empirical mean estimator, leading to the principle known as $P-$ Glivenko-Cantelli. We extend this framework by moving beyond the empirical mean estimator and introducing Uniform Mean Estimability, also called $UME-$ learnability, which captures when a collection permits uniform mean estimation by any arbitrary estimator. We work on the space created by the mean vectors of the collection of distributions. For each distribution, the mean vector records the expected value in each coordinate. We show that separability of the mean vectors is a sufficient condition for $UME-$ learnability. However, we show that separability of the mean vectors is not necessary for $UME-$ learnability by constructing a collection of distributions whose mean vectors are non-separable yet $UME-$ learnable using techniques fundamentally different from those used in our separability-based analysis. Finally, we establish that countable unions of $UME-$ learnable collections are also $UME-$ learnable, solving a conjecture posed in Cohen et al. (2025).

Method: Construct a surrogate model using polynomial chaos expansion fit on a small ensemble of trained flow networks, learning the relationship between reward functions and probability distributions over actions.

Result: The surrogate model enables inexpensive Monte Carlo sampling to estimate policy uncertainty given uncertain rewards, demonstrated on discrete/continuous grid-worlds, symbolic regression, and Bayesian structure learning tasks.

Conclusion: The approach successfully quantifies epistemic uncertainty in generative flow networks through efficient surrogate modeling and Monte Carlo sampling.

Abstract: Generative flow networks are able to sample, via sequential construction, high-reward, complex objects according to a reward function. However, such reward functions are often estimated approximately from noisy data, leading to epistemic uncertainty in the learnt policy. We present an approach to quantify this uncertainty by constructing a surrogate model composed of a polynomial chaos expansion, fit on a small ensemble of trained flow networks. This model learns the relationship between reward functions, parametrised in a low-dimensional space, and the probability distributions over actions at each step along a trajectory of the flow network. The surrogate model can then be used for inexpensive Monte Carlo sampling to estimate the uncertainty in the policy given uncertain rewards. We illustrate the performance of our approach on a discrete and continuous grid-world, symbolic regression, and a Bayesian structure learning task.

Method: Proposes a unified transformer encoder-decoder model trained across multiple problem configurations, using a lightweight adapter mechanism for efficient finetuning to unseen attributes without retraining.

Result: The unified model achieves real-time solutions (1-10 seconds) for networks up to 1,000 nodes, outperforms single-task DRL by 6-14% and traditional optimization by 24-82% in solution quality, with effective finetuning to unseen attributes.

Conclusion: The unified model advances neural combinatorial optimization for time-critical applications, offering computationally efficient, high-quality, and adaptable drone routing for post-disaster road assessment.

Abstract: Post-disaster road assessment (PDRA) is essential for emergency response, enabling rapid evaluation of infrastructure conditions and efficient allocation of resources. Although drones provide a flexible and effective tool for PDRA, routing them in large-scale networks remains challenging. Traditional optimization methods scale poorly and demand domain expertise, while existing deep reinforcement learning (DRL) approaches adopt a single-task paradigm, requiring separate models for each problem variant and lacking adaptability to evolving operational needs. This study proposes a unified model (UM) for drone routing that simultaneously addresses eight PDRA variants. By training a single neural network across multiple problem configurations, UM captures shared structural knowledge while adapting to variant-specific constraints through a modern transformer encoder-decoder architecture. A lightweight adapter mechanism further enables efficient finetuning to unseen attributes without retraining, enhancing deployment flexibility in dynamic disaster scenarios. Extensive experiments demonstrate that the UM reduces training time and parameters by a factor of eight compared with training separate models, while consistently outperforming single-task DRL methods by 6–14% and traditional optimization approaches by 24–82% in terms of solution quality (total collected information value). The model achieves real-time solutions (1–10 seconds) across networks of up to 1,000 nodes, with robustness confirmed through sensitivity analyses. Moreover, finetuning experiments show that unseen attributes can be effectively incorporated with minimal cost while retaining high solution quality. The proposed UM advances neural combinatorial optimization for time-critical applications, offering a computationally efficient, high-quality, and adaptable solution for drone-based PDRA.

Method: Proposed two training methods based on Supervised Fine-tuning and Direct Preference Optimization, evaluated in toxicity reduction testbed. Tested probe accuracy retention through ensemble training, held-out probes, and probe retraining.

Result: Probe-based preference optimization preserves detectability better than classifier methods. Probe diversity provides minimal benefit - simply retraining probes after optimization recovers high detection accuracy.

Conclusion: Probe-based training can be viable for alignment methods, with probe ensembles largely unnecessary when retraining is feasible.

Abstract: Probes trained on model activations can detect undesirable behaviors like deception or biases that are difficult to identify from outputs alone. This makes them useful detectors to identify misbehavior. Furthermore, they are also valuable training signals, since they not only reward outputs, but also good internal processes for arriving at that output. However, training against interpretability tools raises a fundamental concern: when a monitor becomes a training target, it may cease to be reliable (Goodhart’s Law). We propose two methods for training against probes based on Supervised Fine-tuning and Direct Preference Optimization. We conduct an initial exploration of these methods in a testbed for reducing toxicity and evaluate the amount by which probe accuracy drops when training against them. To retain the accuracy of probe-detectors after training, we attempt (1) to train against an ensemble of probes, (2) retain held-out probes that aren’t used for training, and (3) retrain new probes after training. First, probe-based preference optimization unexpectedly preserves probe detectability better than classifier-based methods, suggesting the preference learning objective incentivizes maintaining rather than obfuscating relevant representations. Second, probe diversity provides minimal practical benefit - simply retraining probes after optimization recovers high detection accuracy. Our findings suggest probe-based training can be viable for certain alignment methods, though probe ensembles are largely unnecessary when retraining is feasible.

Method: Proposes Spatiotemporal Region Distortion (SRD) method that creates synthetic anomalies by applying severe augmentations to localized regions in normal videos while maintaining temporal consistency, then integrates these with noisy pseudo-labels to reduce label noise for effective adversarial training.

Result: The method significantly enhances WSVAD model robustness against adversarial attacks, outperforming state-of-the-art methods by an average of 71.0% in overall AUROC performance across multiple benchmarks.

Conclusion: The SRD method successfully addresses the limitations of weak supervision in WSVAD by generating precisely annotated synthetic anomalies, enabling effective adversarial training and substantially improving model robustness against attacks.

Abstract: Weakly Supervised Video Anomaly Detection (WSVAD) has achieved notable advancements, yet existing models remain vulnerable to adversarial attacks, limiting their reliability. Due to the inherent constraints of weak supervision, where only video-level labels are provided despite the need for frame-level predictions, traditional adversarial defense mechanisms, such as adversarial training, are not effective since video-level adversarial perturbations are typically weak and inadequate. To address this limitation, pseudo-labels generated directly from the model can enable frame-level adversarial training; however, these pseudo-labels are inherently noisy, significantly degrading performance. We therefore introduce a novel Pseudo-Anomaly Generation method called Spatiotemporal Region Distortion (SRD), which creates synthetic anomalies by applying severe augmentations to localized regions in normal videos while preserving temporal consistency. Integrating these precisely annotated synthetic anomalies with the noisy pseudo-labels substantially reduces label noise, enabling effective adversarial training. Extensive experiments demonstrate that our method significantly enhances the robustness of WSVAD models against adversarial attacks, outperforming state-of-the-art methods by an average of 71.0% in overall AUROC performance across multiple benchmarks. The implementation and code are publicly available at https://github.com/rohban-lab/FrameShield.

Method: Proposes Excision Score (ES) which computes longest common subsequence (LCS) to remove content shared between the original document and both ground truth and predicted revisions, then compares only the remaining divergent regions. Uses approximation to speed LCS computation from cubic to quadratic time.

Result: ES significantly outperforms existing measures in code-editing evaluation. On HumanEvalFix, ES improves over SARI by 12% Pearson correlation and >21% over standard measures like BLEU. With increased shared context, ES’ improvement over SARI increases to 20% and >30% over standard measures. ES also handles corner cases like moved code blocks and properly rewards matching insertions/deletions.

Conclusion: Excision Score provides a more accurate and robust measure for evaluating document revisions by focusing exclusively on the changed content, aligning better with human judgment and addressing fundamental flaws in existing similarity metrics.

Abstract: Many tasks revolve around editing a document, whether code or text. We formulate the revision similarity problem to unify a wide range of machine learning evaluation problems whose goal is to assess a revision to an existing document. We observe that revisions usually change only a small portion of an existing document, so the existing document and its immediate revisions share a majority of their content. We formulate five adequacy criteria for revision similarity measures, designed to align them with human judgement. We show that popular pairwise measures, like BLEU, fail to meet these criteria, because their scores are dominated by the shared content. They report high similarity between two revisions when humans would assess them as quite different. This is a fundamental flaw we address. We propose a novel static measure, Excision Score (ES), which computes longest common subsequence (LCS) to remove content shared by an existing document with the ground truth and predicted revisions, before comparing only the remaining divergent regions. This is analogous to a surgeon creating a sterile field to focus on the work area. We use approximation to speed the standard cubic LCS computation to quadratic. In code-editing evaluation, where static measures are often used as a cheap proxy for passing tests, we demonstrate that ES surpasses existing measures. When aligned with test execution on HumanEvalFix, ES improves over its nearest competitor, SARI, by 12% Pearson correlation and by >21% over standard measures like BLEU. The key criterion is invariance to shared context; when we perturb HumanEvalFix with increased shared context, ES’ improvement over SARI increases to 20% and >30% over standard measures. ES also handles other corner cases that other measures do not, such as correctly aligning moved code blocks, and appropriately rewarding matching insertions or deletions.

Method: Hierarchical SAGIN-MEC architecture with coordination between UDs, UAVs, and satellites. Uses MADDPG for continuous temporal decisions and COCG (convex optimization and coalitional game) to handle hybrid/varying-dimensional decisions deterministically.

Result: Significantly improves user-centric performance (aggregated UD cost, task completion delay, UD energy consumption) with slight UAV energy increase. Shows superior convergence stability and scalability compared to benchmarks.

Conclusion: MADDPG-COCG effectively addresses the NP-hard optimization problem in SAGIN-MEC systems, providing a robust solution for low-altitude economy applications with enhanced performance and stability.

Abstract: Space-air-ground integrated multi-access edge computing (SAGIN-MEC) provides a promising solution for the rapidly developing low-altitude economy (LAE) to deliver flexible and wide-area computing services. However, fully realizing the potential of SAGIN-MEC in the LAE presents significant challenges, including coordinating decisions across heterogeneous nodes with different roles, modeling complex factors such as mobility and network variability, and handling real-time decision-making under partially observable environment with hybrid variables. To address these challenges, we first present a hierarchical SAGIN-MEC architecture that enables the coordination between user devices (UDs), uncrewed aerial vehicles (UAVs), and satellites. Then, we formulate a UD cost minimization optimization problem (UCMOP) to minimize the UD cost by jointly optimizing the task offloading ratio, UAV trajectory planning, computing resource allocation, and UD association. We show that the UCMOP is an NP-hard problem. To overcome this challenge, we propose a multi-agent deep deterministic policy gradient (MADDPG)-convex optimization and coalitional game (MADDPG-COCG) algorithm. Specifically, we employ the MADDPG algorithm to optimize the continuous temporal decisions for heterogeneous nodes in the partially observable SAGIN-MEC system. Moreover, we propose a convex optimization and coalitional game (COCG) method to enhance the conventional MADDPG by deterministically handling the hybrid and varying-dimensional decisions. Simulation results demonstrate that the proposed MADDPG-COCG algorithm significantly enhances the user-centric performances in terms of the aggregated UD cost, task completion delay, and UD energy consumption, with a slight increase in UAV energy consumption, compared to the benchmark algorithms. Moreover, the MADDPG-COCG algorithm shows superior convergence stability and scalability.

Method: Uses LLM-assisted, expert-validated process to curate structured positive/negative clinical observations, then leverages pre-trained multimodal model to align ECG and text embeddings for zero-shot classification without disease-specific fine-tuning.

Result: ZETA demonstrates competitive zero-shot classification performance with enhanced interpretability, grounding predictions in specific clinically relevant positive and negative diagnostic features.

Conclusion: Aligning ECG analysis with structured clinical knowledge enables building more transparent, generalizable, and trustworthy AI diagnostic systems, with potential for broader adoption in clinical practice.

Abstract: Electrocardiogram (ECG) interpretation is essential for cardiovascular disease diagnosis, but current automated systems often struggle with transparency and generalization to unseen conditions. To address this, we introduce ZETA, a zero-shot multimodal framework designed for interpretable ECG diagnosis aligned with clinical workflows. ZETA uniquely compares ECG signals against structured positive and negative clinical observations, which are curated through an LLM-assisted, expert-validated process, thereby mimicking differential diagnosis. Our approach leverages a pre-trained multimodal model to align ECG and text embeddings without disease-specific fine-tuning. Empirical evaluations demonstrate ZETA’s competitive zero-shot classification performance and, importantly, provide qualitative and quantitative evidence of enhanced interpretability, grounding predictions in specific, clinically relevant positive and negative diagnostic features. ZETA underscores the potential of aligning ECG analysis with structured clinical knowledge for building more transparent, generalizable, and trustworthy AI diagnostic systems. We will release the curated observation dataset and code to facilitate future research.

Method: GNNs were pretrained on 24 classical algorithms from the CLRS Algorithmic Reasoning Benchmark, then used to initialize and freeze selected layers of a second GNN for molecular property prediction on ogbg-molhiv and ogbg-molclintox datasets.

Result: Pretrained models consistently outperformed or tied with randomly initialized baselines. Segments Intersect algorithm pretraining achieved 6% absolute gain on ogbg-molhiv, and Dijkstra pretraining achieved 3% gain on ogbg-molclintox.

Conclusion: Embedding classical algorithmic priors into GNNs provides useful inductive biases that boost performance on complex, real-world graph data, bridging the gap between neural network flexibility and algorithmic correctness guarantees.

Abstract: Neural networks excel at processing unstructured data but often fail to generalise out-of-distribution, whereas classical algorithms guarantee correctness but lack flexibility. We explore whether pretraining Graph Neural Networks (GNNs) on classical algorithms can improve their performance on molecular property prediction tasks from the Open Graph Benchmark: ogbg-molhiv (HIV inhibition) and ogbg-molclintox (clinical toxicity). GNNs trained on 24 classical algorithms from the CLRS Algorithmic Reasoning Benchmark are used to initialise and freeze selected layers of a second GNN for molecular prediction. Compared to a randomly initialised baseline, the pretrained models achieve consistent wins or ties, with the Segments Intersect algorithm pretraining yielding a 6% absolute gain on ogbg-molhiv and Dijkstra pretraining achieving a 3% gain on ogbg-molclintox. These results demonstrate embedding classical algorithmic priors into GNNs provides useful inductive biases, boosting performance on complex, real-world graph data.

Method: Developed Gride method for intrinsic dimension estimation without data decimation; studied probability density evolution across hidden layers in state-of-the-art networks; analyzed generalization in wide neural networks with redundant representations.

Result: Found that initial layers create unimodal probability density removing irrelevant structure, while subsequent layers develop hierarchical density peaks mirroring semantic concepts; showed that wide networks learn redundant representations rather than overfitting when regularized with zero training error.

Conclusion: Deep neural networks develop hierarchical semantic representations through probability density evolution, and generalization improves through redundant representations in wide networks rather than classical overfitting, providing new insights into neural network internal mechanisms.

Abstract: The goal of this thesis is to improve our understanding of the internal mechanisms by which deep artificial neural networks create meaningful representations and are able to generalize. We focus on the challenge of characterizing the semantic content of the hidden representations with unsupervised learning tools, partially developed by us and described in this thesis, which allow harnessing the low-dimensional structure of the data. Chapter 2. introduces Gride, a method that allows estimating the intrinsic dimension of the data as an explicit function of the scale without performing any decimation of the data set. Our approach is based on rigorous distributional results that enable the quantification of uncertainty of the estimates. Moreover, our method is simple and computationally efficient since it relies only on the distances among nearest data points. In Chapter 3, we study the evolution of the probability density across the hidden layers in some state-of-the-art deep neural networks. We find that the initial layers generate a unimodal probability density getting rid of any structure irrelevant to classification. In subsequent layers, density peaks arise in a hierarchical fashion that mirrors the semantic hierarchy of the concepts. This process leaves a footprint in the probability density of the output layer, where the topography of the peaks allows reconstructing the semantic relationships of the categories. In Chapter 4, we study the problem of generalization in deep neural networks: adding parameters to a network that interpolates its training data will typically improve its generalization performance, at odds with the classical bias-variance trade-off. We show that wide neural networks learn redundant representations instead of overfitting to spurious correlation and that redundant neurons appear only if the network is regularized and the training error is zero.

Method: Introduces 4D positional encoding for arbitrary EEG signals, uses masked autoencoding objective, and pretrains on 60,000+ hours of EEG data from 92 datasets across 25,000 subjects.

Result: Achieves state-of-the-art performance on 10 downstream EEG tasks including motor imagery, seizure detection, sleep staging, cognitive load estimation, and emotion recognition with minimal fine-tuning.

Conclusion: REVE enables standardized EEG research and accelerates clinical neuroscience progress through strong generalization and nuanced spatio-temporal modeling capabilities.

Abstract: Foundation models have transformed AI by reducing reliance on task-specific data through large-scale pretraining. While successful in language and vision, their adoption in EEG has lagged due to the heterogeneity of public datasets, which are collected under varying protocols, devices, and electrode configurations. Existing EEG foundation models struggle to generalize across these variations, often restricting pretraining to a single setup, resulting in suboptimal performance, in particular under linear probing. We present REVE (Representation for EEG with Versatile Embeddings), a pretrained model explicitly designed to generalize across diverse EEG signals. REVE introduces a novel 4D positional encoding scheme that enables it to process signals of arbitrary length and electrode arrangement. Using a masked autoencoding objective, we pretrain REVE on over 60,000 hours of EEG data from 92 datasets spanning 25,000 subjects, representing the largest EEG pretraining effort to date. REVE achieves state-of-the-art results on 10 downstream EEG tasks, including motor imagery classification, seizure detection, sleep staging, cognitive load estimation, and emotion recognition. With little to no fine-tuning, it demonstrates strong generalization, and nuanced spatio-temporal modeling. We release code, pretrained weights, and tutorials to support standardized EEG research and accelerate progress in clinical neuroscience.

Method: HOPSS uses base solution functions from reliable solvers, aligns them via downsampling, and applies homologous perturbation by combining two solution functions with random noise to generate comparable-precision PDE data points.

Result: HOPSS reduces time complexity significantly - generating 10,000 samples for Navier-Stokes equation in approximately 10% of traditional methods’ time while achieving comparable model training performance.

Conclusion: The proposed HOPSS algorithm effectively accelerates dataset generation for neural operators while preserving training precision, offering a more efficient alternative to traditional data generation methods.

Abstract: Data-driven deep learning methods like neural operators have advanced in solving nonlinear temporal partial differential equations (PDEs). However, these methods require large quantities of solution pairs\u2014the solution functions and right-hand sides (RHS) of the equations. These pairs are typically generated via traditional numerical methods, which need thousands of time steps iterations far more than the dozens required for training, creating heavy computational and temporal overheads. To address these challenges, we propose a novel data generation algorithm, called HOmologous Perturbation in Solution Space (HOPSS), which directly generates training datasets with fewer time steps rather than following the traditional approach of generating large time steps datasets. This algorithm simultaneously accelerates dataset generation and preserves the approximate precision required for model training. Specifically, we first obtain a set of base solution functions from a reliable solver, usually with thousands of time steps, and then align them in time steps with training datasets by downsampling. Subsequently, we propose a “homologous perturbation” approach: by combining two solution functions (one as the primary function, the other as a homologous perturbation term scaled by a small scalar) with random noise, we efficiently generate comparable-precision PDE data points. Finally, using these data points, we compute the variation in the original equation’s RHS to form new solution pairs. Theoretical and experimental results show HOPSS lowers time complexity. For example, on the Navier-Stokes equation, it generates 10,000 samples in approximately 10% of traditional methods’ time, with comparable model training performance.

Method: Developed a general framework for exact SHAP computation on Tensor Networks, with special focus on Tensor Train structures enabling parallel computation in poly-logarithmic time.

Result: SHAP computation can be performed efficiently for Tensor Trains and generalized to other ML models like decision trees, ensembles, linear models, and RNNs. For binarized neural networks, SHAP becomes tractable with fixed width but remains hard with constant depth.

Conclusion: Width rather than depth is the primary computational bottleneck for SHAP computation in neural networks, and Tensor Network representations enable efficient exact SHAP computation for various ML models.

Abstract: Although Shapley additive explanations (SHAP) can be computed in polynomial time for simple models like decision trees, they unfortunately become NP-hard to compute for more expressive black-box models like neural networks - where generating explanations is often most critical. In this work, we analyze the problem of computing SHAP explanations for Tensor Networks (TNs), a broader and more expressive class of models than those for which current exact SHAP algorithms are known to hold, and which is widely used for neural network abstraction and compression. First, we introduce a general framework for computing provably exact SHAP explanations for general TNs with arbitrary structures. Interestingly, we show that, when TNs are restricted to a Tensor Train (TT) structure, SHAP computation can be performed in poly-logarithmic time using parallel computation. Thanks to the expressiveness power of TTs, this complexity result can be generalized to many other popular ML models such as decision trees, tree ensembles, linear models, and linear RNNs, therefore tightening previously reported complexity results for these families of models. Finally, by leveraging reductions of binarized neural networks to Tensor Network representations, we demonstrate that SHAP computation can become efficiently tractable when the network’s width is fixed, while it remains computationally hard even with constant depth. This highlights an important insight: for this class of models, width - rather than depth - emerges as the primary computational bottleneck in SHAP computation.

Method: Proposed Generalised Flow Maps (GFM) with three self-distillation-based training methods: Generalised Lagrangian Flow Maps, Generalised Eulerian Flow Maps, and Generalised Progressive Flow Maps.

Result: GFMs achieve state-of-the-art sample quality for single- and few-step evaluations, and superior or competitive log-likelihoods on geometric datasets including geospatial data, RNA torsion angles, and hyperbolic manifolds.

Conclusion: GFMs unify and elevate existing Euclidean few-step generative models to Riemannian setting while providing efficient inference with few steps.

Abstract: Geometric data and purpose-built generative models on them have become ubiquitous in high-impact deep learning application domains, ranging from protein backbone generation and computational chemistry to geospatial data. Current geometric generative models remain computationally expensive at inference – requiring many steps of complex numerical simulation – as they are derived from dynamical measure transport frameworks such as diffusion and flow-matching on Riemannian manifolds. In this paper, we propose Generalised Flow Maps (GFM), a new class of few-step generative models that generalises the Flow Map framework in Euclidean spaces to arbitrary Riemannian manifolds. We instantiate GFMs with three self-distillation-based training methods: Generalised Lagrangian Flow Maps, Generalised Eulerian Flow Maps, and Generalised Progressive Flow Maps. We theoretically show that GFMs, under specific design decisions, unify and elevate existing Euclidean few-step generative models, such as consistency models, shortcut models, and meanflows, to the Riemannian setting. We benchmark GFMs against other geometric generative models on a suite of geometric datasets, including geospatial data, RNA torsion angles, and hyperbolic manifolds, and achieve state-of-the-art sample quality for single- and few-step evaluations, and superior or competitive log-likelihoods using the implicit probability flow.

Method: Theoretical analysis of clustering limits under PABM, which introduces separate popularity parameters for intra- and inter-cluster connections, and numerical experiments using spectral clustering with k² eigenvectors.

Result: Characterized optimal error rate for clustering under PABM, showing cluster recovery remains possible even when traditional edge-density signals vanish, and demonstrated that spectral clustering with k² eigenvectors outperforms traditional approaches.

Conclusion: PABM captures a dimension of degree heterogeneity overlooked by DCBM, where local connectivity differences can enhance cluster separability independently of global edge densities, requiring spectral embeddings with k² eigenvectors for optimal clustering.

Abstract: This paper establishes the theoretical limits of graph clustering under the Popularity-Adjusted Block Model (PABM), addressing limitations of existing models. In contrast to the Stochastic Block Model (SBM), which assumes uniform vertex degrees, and to the Degree-Corrected Block Model (DCBM), which applies uniform degree corrections across clusters, PABM introduces separate popularity parameters for intra- and inter-cluster connections. Our main contribution is the characterization of the optimal error rate for clustering under PABM, which provides novel insights on clustering hardness: we demonstrate that unlike SBM and DCBM, cluster recovery remains possible in PABM even when traditional edge-density signals vanish, provided intra- and inter-cluster popularity coefficients differ. This highlights a dimension of degree heterogeneity captured by PABM but overlooked by DCBM: local differences in connectivity patterns can enhance cluster separability independently of global edge densities. Finally, because PABM exhibits a richer structure, its expected adjacency matrix has rank between $k$ and $k^2$, where $k$ is the number of clusters. As a result, spectral embeddings based on the top $k$ eigenvectors may fail to capture important structural information. Our numerical experiments on both synthetic and real datasets confirm that spectral clustering algorithms incorporating $k^2$ eigenvectors outperform traditional spectral approaches.

Method: Developed theoretical bounds on uncertainty calibration errors (ECE and ENCE) under different equivariance conditions, complemented by numerical experiments on real and simulated datasets analyzing symmetry mismatch, group size, and uncertainty types.

Result: Proved lower and upper bounds on calibration errors, demonstrating that equivariance affects model generalization limits and symmetry mismatch leads to miscalibration in both classification and regression tasks.

Conclusion: The work establishes a theoretical foundation connecting equivariance to uncertainty estimation, revealing how symmetry properties impact model calibration and providing insights for improving uncertainty-aware equivariant models.

Abstract: Data-sparse settings such as robotic manipulation, molecular physics, and galaxy morphology classification are some of the hardest domains for deep learning. For these problems, equivariant networks can help improve modeling across undersampled parts of the input space, and uncertainty estimation can guard against overconfidence. However, until now, the relationships between equivariance and model confidence, and more generally equivariance and model calibration, has yet to be studied. Since traditional classification and regression error terms show up in the definitions of calibration error, it is natural to suspect that previous work can be used to help understand the relationship between equivariance and calibration error. In this work, we present a theory relating equivariance to uncertainty estimation. By proving lower and upper bounds on uncertainty calibration errors (ECE and ENCE) under various equivariance conditions, we elucidate the generalization limits of equivariant models and illustrate how symmetry mismatch can result in miscalibration in both classification and regression. We complement our theoretical framework with numerical experiments that clarify the relationship between equivariance and uncertainty using a variety of real and simulated datasets, and we comment on trends with symmetry mismatch, group size, and aleatoric and epistemic uncertainties.

Method: Train a pointer network with reinforcement learning on 100-node TSP instances, then fit sparse autoencoders to the encoder’s residual stream to discover an overcomplete dictionary of interpretable features.

Result: The analysis reveals the solver naturally develops features mirroring fundamental TSP concepts: boundary detectors for convex-hull nodes, cluster-sensitive features for dense regions, and separator features for geometric partitions.

Conclusion: This provides the first model-internal account of what neural TSP solvers compute, demonstrates geometric structure emerges without supervision, and suggests pathways for transparent hybrid systems combining neural efficiency with algorithmic interpretability.

Abstract: Neural networks have advanced combinatorial optimization, with Transformer-based solvers achieving near-optimal solutions on the Traveling Salesman Problem (TSP) in milliseconds. However, these models operate as black boxes, providing no insight into the geometric patterns they learn or the heuristics they employ during tour construction. We address this opacity by applying sparse autoencoders (SAEs), a mechanistic interpretability technique, to a Transformer-based TSP solver, representing the first application of activation-based interpretability methods to operations research models. We train a pointer network with reinforcement learning on 100-node instances, then fit an SAE to the encoder’s residual stream to discover an overcomplete dictionary of interpretable features. Our analysis reveals that the solver naturally develops features mirroring fundamental TSP concepts: boundary detectors that activate on convex-hull nodes, cluster-sensitive features responding to locally dense regions, and separator features encoding geometric partitions. These findings provide the first model-internal account of what neural TSP solvers compute before node selection, demonstrate that geometric structure emerges without explicit supervision, and suggest pathways toward transparent hybrid systems that combine neural efficiency with algorithmic interpretability. Interactive feature explorer: https://reubennarad.github.io/TSP_interp

Method: Jointly learns a latent space and group representation where group actions correspond to invertible linear maps, using contrastive learning from observation pairs (y, g·y).

Result: High-fidelity equivariance with group operations faithfully reproduced in latent space, validated on finite groups (discrete rotations and translations) and non-abelian groups (O(n), GL(n)).

Conclusion: First successful demonstration of general-purpose encoder-only equivariant learning from group action observations alone, including non-trivial non-abelian groups and product groups.

Abstract: We propose Equivariance by Contrast (EbC) to learn equivariant embeddings from observation pairs $(\mathbf{y}, g \cdot \mathbf{y})$, where $g$ is drawn from a finite group acting on the data. Our method jointly learns a latent space and a group representation in which group actions correspond to invertible linear maps – without relying on group-specific inductive biases. We validate our approach on the infinite dSprites dataset with structured transformations defined by the finite group $G:= (R_m \times \mathbb{Z}_n \times \mathbb{Z}_n)$, combining discrete rotations and periodic translations. The resulting embeddings exhibit high-fidelity equivariance, with group operations faithfully reproduced in latent space. On synthetic data, we further validate the approach on the non-abelian orthogonal group $O(n)$ and the general linear group $GL(n)$. We also provide a theoretical proof for identifiability. While broad evaluation across diverse group types on real-world data remains future work, our results constitute the first successful demonstration of general-purpose encoder-only equivariant learning from group action observations alone, including non-trivial non-abelian groups and a product group motivated by modeling affine equivariances in computer vision.

Method: Proposes ViTime framework that transforms time series forecasting from numerical fitting to operations in a binary image-based metric space. Includes RealTS algorithm for generating realistic synthetic training samples to enhance model generalizability.

Result: ViTime achieves state-of-the-art performance: 9-15% improvement over TimesFM in zero-shot scenarios, surpasses both foundation models and supervised benchmarks with minimal fine-tuning (10% data), and shows 20-30% better robustness under data perturbations compared to TimesFM.

Conclusion: ViTime represents a paradigm shift in time series forecasting by leveraging visual space data operations, demonstrating superior performance, robustness, and generalizability as a foundation model for various TSF applications.

Abstract: Time series forecasting (TSF) possesses great practical values in various fields, including power and energy, transportation, etc. TSF methods have been studied based on knowledge from classical statistics to modern deep learning. Yet, all of them were developed based on one fundamental concept, the numerical data fitting. Thus, the models developed have long been known to be problem-specific and lacking application generalizability. Practitioners expect a TSF foundation model that serves TSF tasks in different applications. The central question is then how to develop such a TSF foundation model. This paper offers one pioneering study in the TSF foundation model development method and proposes a vision intelligence-powered framework, ViTime, for the first time. ViTime fundamentally shifts TSF from numerical fitting to operations based on a binary image-based time series metric space and naturally supports both point and probabilistic forecasting. We also provide rigorous theoretical analyses of ViTime, including quantization-induced system error bounds and principled strategies for optimal parameter selection. Furthermore, we propose RealTS, an innovative synthesis algorithm generating diverse and realistic training samples, effectively enriching the training data and significantly enhancing model generalizability. Extensive experiments demonstrate ViTime’s state-of-the-art performance. In zero-shot scenarios, ViTime outperforms TimesFM by 9-15%. With just 10% fine-tuning data, ViTime surpasses both leading foundation models and fully-supervised benchmarks, a gap that widens with 100% fine-tuning. ViTime also exhibits exceptional robustness, effectively handling missing data and outperforming TimesFM by 20-30% under various data perturbations, validating the power of its visual space data operation paradigm.

Method: Axiomatic training where systems learn from multiple demonstrations of causal axioms (transitivity and d-separation), training transformer models from scratch to avoid data contamination, and extending the method to finetune language models like Llama-3-8B-Instruct.

Result: Models trained on linear causal chains generalize well to complex graphs including longer chains, reversed order chains, and branching graphs. Finetuning language models leads to significant gains on causal benchmarks, achieving state-of-the-art performance surpassing GPT-4 on some tasks.

Conclusion: Axiomatic training enables effective learning of causal reasoning from symbolic demonstrations, allowing generalization to complex scenarios and achieving strong performance on causal benchmarks.

Abstract: For text-based AI systems to interact in the real world, causal reasoning is an essential skill. Since active interventions are costly, we study to what extent a system can learn causal reasoning from symbolic demonstrations of causal axioms. Specifically, we present an axiomatic training method where the system learns from multiple demonstrations of a causal axiom (or rule), rather than incorporating the axiom as an inductive bias or inferring it from data values. A key question is whether the system would learn to generalize from the axiom demonstrations to more complex scenarios. Our results, based on applying axiomatic training to learn the transitivity axiom and d-separation rule, indicate that such generalization is possible. To avoid data contamination issues, we start with a 67 million parameter transformer model and train it from scratch. On both tasks, we find that a model trained on linear causal chains (along with some noisy variations) can generalize well to complex graphs, including longer causal chains, causal chains with reversed order, and graphs with branching.To handle diverse text inputs, the same method is extended to finetune language models. Finetuning Llama-3-8B-Instruct model on our axiomatic data leads to significant gains on causal benchmarks such as Corr2Cause and CLEAR, in some cases providing state-of-the-art performance surpassing GPT-4.

Method: The authors use a new proof technique building on gradient domination for policy gradient convergence in standard RL. They analyze both tabular settings and large state-action spaces, where they approximate occupancy measures using maximum likelihood estimation within function approximation classes.

Result: The paper provides global optimality guarantees for PG methods in RLGU, with sample complexity that scales only with the dimension of the approximation class rather than the full state-action space size in large-scale settings.

Conclusion: The work successfully establishes theoretical foundations for policy gradient methods in RLGU, opening avenues for analyzing various policy parameterizations and extending applicability beyond tabular settings to large-scale problems.

Abstract: Reinforcement learning with general utilities (RLGU) offers a unifying framework to capture several problems beyond standard expected returns, including imitation learning, pure exploration, and safe RL. Despite recent fundamental advances in the theoretical analysis of policy gradient (PG) methods for standard RL and recent efforts in RLGU, the understanding of these PG algorithms and their scope of application in RLGU still remain limited. In this work, we establish global optimality guarantees of PG methods for RLGU in which the objective is a general concave utility function of the state-action occupancy measure. In the tabular setting, we provide global optimality results using a new proof technique building on recent theoretical developments on the convergence of PG methods for standard RL using gradient domination. Our proof technique opens avenues for analyzing policy parameterizations beyond the direct policy parameterization for RLGU. In addition, we provide global optimality results for large state-action space settings beyond prior work which has mostly focused on the tabular setting. In this large scale setting, we adapt PG methods by approximating occupancy measures within a function approximation class using maximum likelihood estimation. Our sample complexity only scales with the dimension induced by our approximation class instead of the size of the state-action space.

Method: View linear attention as linear predictor in RKHS, convert learning problem to ordinary linear predictor in expanded feature space, and efficiently identify training datasets for guaranteed generalization.

Result: Proved polynomial-time learnability (strong agnostic PAC learning) for linear Transformers, with empirical validation on learning random linear attention networks, key-value associations, and finite automata.

Conclusion: Flexible and general models of computation including associative memories, finite automata, and bounded UTMs are efficiently learnable via linear attention, bridging expressivity and learnability gap.

Abstract: Previous research has explored the computational expressivity of Transformer models in simulating Boolean circuits or Turing machines. However, the learnability of these simulators from observational data has remained an open question. Our study addresses this gap by providing the first polynomial-time learnability results (specifically strong, agnostic PAC learning) for single-layer Transformers with linear attention. We show that linear attention may be viewed as a linear predictor in a suitably defined RKHS. As a consequence, the problem of learning any linear transformer may be converted into the problem of learning an ordinary linear predictor in an expanded feature space, and any such predictor may be converted back into a multiheaded linear transformer. Moving to generalization, we show how to efficiently identify training datasets for which every empirical risk minimizer is equivalent (up to trivial symmetries) to the linear Transformer that generated the data, thereby guaranteeing the learned model will correctly generalize across all inputs. Finally, we provide examples of computations expressible via linear attention and therefore polynomial-time learnable, including associative memories, finite automata, and a class of Universal Turing Machine (UTMs) with polynomially bounded computation histories. We empirically validate our theoretical findings on three tasks: learning random linear attention networks, key–value associations, and learning to execute finite automata. Our findings bridge a critical gap between theoretical expressivity and learnability of Transformers, and show that flexible and general models of computation are efficiently learnable.

Method: A guide network steers a target network using neural distance function. Target minimizes task loss plus layerwise representational similarity against frozen guide. Works with both trained and untrained guides.

Result: Guidance prevents FCN overfitting on ImageNet, narrows RNN-Transformer gap, boosts plain CNNs toward ResNet accuracy, and helps Transformers on RNN-favored tasks. Guidance-driven initialization alone can mitigate FCN overfitting.

Conclusion: Guidance provides a mathematical tool to investigate architectural priors and could automate architecture design in the long term.

Abstract: We demonstrate that architectures which traditionally are considered to be ill-suited for a task can be trained using inductive biases from another architecture. We call a network untrainable when it overfits, underfits, or converges to poor results even when tuning their hyperparameters. For example, fully connected networks overfit on object recognition while deep convolutional networks without residual connections underfit. The traditional answer is to change the architecture to impose some inductive bias, although the nature of that bias is unknown. We introduce guidance, where a guide network steers a target network using a neural distance function. The target minimizes its task loss plus a layerwise representational similarity against the frozen guide. If the guide is trained, this transfers over the architectural prior and knowledge of the guide to the target. If the guide is untrained, this transfers over only part of the architectural prior of the guide. We show that guidance prevents FCN overfitting on ImageNet, narrows the vanilla RNN-Transformer gap, boosts plain CNNs toward ResNet accuracy, and aids Transformers on RNN-favored tasks. We further identify that guidance-driven initialization alone can mitigate FCN overfitting. Our method provides a mathematical tool to investigate priors and architectures, and in the long term, could automate architecture design.

Method: The survey reviews deep learning methodologies including audio, text, and visual processing for cognitive decline detection, covering state-of-the-art approaches like Transformer architecture and foundation models, as well as multimodal model integration.

Result: Text-based approaches consistently outperform other modalities in most cases. Combining different modalities into multimodal models consistently enhances performance across nearly all scenarios.

Conclusion: Non-intrusive deep learning methods are effective for cognitive decline detection, with text-based approaches showing superior performance and multimodal integration providing consistent improvements in detection accuracy.

Abstract: Cognitive decline is a natural part of aging. However, under some circumstances, this decline is more pronounced than expected, typically due to disorders such as Alzheimer’s disease. Early detection of an anomalous decline is crucial, as it can facilitate timely professional intervention. While medical data can help, it often involves invasive procedures. An alternative approach is to employ non-intrusive techniques such as speech or handwriting analysis, which do not disturb daily activities. This survey reviews the most relevant non-intrusive methodologies that use deep learning techniques to automate the cognitive decline detection task, including audio, text, and visual processing. We discuss the key features and advantages of each modality and methodology, including state-of-the-art approaches like Transformer architecture and foundation models. In addition, we present studies that integrate different modalities to develop multimodal models. We also highlight the most significant datasets and the quantitative results from studies using these resources. From this review, several conclusions emerge. In most cases, text-based approaches consistently outperform other modalities. Furthermore, combining various approaches from individual modalities into a multimodal model consistently enhances performance across nearly all scenarios.

Method: Proposed ONI - a distributed architecture that simultaneously learns RL policy and intrinsic reward function using LLM feedback. Uses asynchronous LLM server to annotate collected experience, then distills into intrinsic reward model. Explored hashing, classification, and ranking models for reward modeling.

Result: Achieves state-of-the-art performance across challenging NetHack Learning Environment tasks while removing the need for large offline datasets required by prior work.

Conclusion: ONI addresses scalability and data dependency limitations of previous approaches through algorithmic and systems-level innovations, enabling effective reward synthesis from natural language descriptions.

Abstract: Automatically synthesizing dense rewards from natural language descriptions is a promising paradigm in reinforcement learning (RL), with applications to sparse reward problems, open-ended exploration, and hierarchical skill design. Recent works have made promising steps by exploiting the prior knowledge of large language models (LLMs). However, these approaches suffer from important limitations: they are either not scalable to problems requiring billions of environment samples, due to requiring LLM annotations for each observation, or they require a diverse offline dataset, which may not exist or be impossible to collect. In this work, we address these limitations through a combination of algorithmic and systems-level contributions. We propose ONI, a distributed architecture that simultaneously learns an RL policy and an intrinsic reward function using LLM feedback. Our approach annotates the agent’s collected experience via an asynchronous LLM server, which is then distilled into an intrinsic reward model. We explore a range of algorithmic choices for reward modeling with varying complexity, including hashing, classification, and ranking models. Our approach achieves state-of-the-art performance across a range of challenging tasks from the NetHack Learning Environment, while removing the need for large offline datasets required by prior work. We make our code available at https://github.com/facebookresearch/oni.

Method: The study investigates Adam’s sensitivity by testing its performance under random rotations of parameter space in transformer training. It also identifies structured rotations that preserve/enhance performance and examines existing rotation-dependent assumptions in literature.

Result: Adam’s performance degrades under random rotations but can be preserved or enhanced with structured rotations. Existing rotation-dependent assumptions fail to explain Adam’s behavior across rotation types. Orthogonality of updates emerges as a promising indicator of Adam’s basis sensitivity.

Conclusion: Orthogonality of the update is a key quantity for developing rotation-dependent theoretical frameworks that better explain Adam’s empirical success, as conventional rotation-invariant assumptions are insufficient.

Abstract: Despite its widespread adoption, Adam’s advantage over Stochastic Gradient Descent (SGD) lacks a comprehensive theoretical explanation. This paper investigates Adam’s sensitivity to rotations of the parameter space. We observe that Adam’s performance in training transformers degrades under random rotations of the parameter space, indicating a crucial sensitivity to the choice of basis in practice. This reveals that conventional rotation-invariant assumptions are insufficient to capture Adam’s advantages theoretically. To better understand the rotation-dependent properties that benefit Adam, we also identify structured rotations that preserve or even enhance its empirical performance. We then examine the rotation-dependent assumptions in the literature and find that they fall short in explaining Adam’s behaviour across various rotation types. In contrast, we verify the orthogonality of the update as a promising indicator of Adam’s basis sensitivity, suggesting it may be the key quantity for developing rotation-dependent theoretical frameworks that better explain its empirical success.

Method: GoRA leverages gradient information during training to dynamically assign optimal ranks and initialize low-rank adapter weights in an adaptive manner within a single unified framework.

Result: Extensive experiments show GoRA consistently outperforms existing LoRA-based methods while preserving vanilla LoRA’s efficiency. For Llama3.1-8B-Base mathematical reasoning, it achieves 5.13-point improvement over standard LoRA and even outperforms full fine-tuning by 2.05 points under high-rank settings.

Conclusion: GoRA is the first method that simultaneously addresses rank selection and initialization limitations in a unified framework, enabling more effective and efficient adaptation of large language models.

Abstract: Low-Rank Adaptation (LoRA) is a crucial method for efficiently fine-tuning large language models (LLMs), with its effectiveness influenced by two key factors: rank selection and weight initialization. While numerous LoRA variants have been proposed to improve performance by addressing one of these aspects, they often compromise usability or computational efficiency. In this paper, we analyze and identify the core limitations of existing approaches and propose a novel framework–GoRA (Gradient-driven Adaptive Low Rank Adaptation)–that simultaneously adapts both the rank and initialization strategy within a unified framework. GoRA leverages gradient information during training to dynamically assign optimal ranks and initialize low-rank adapter weights in an adaptive manner. To our knowledge, GoRA is the first method that not only addresses the limitations of prior approaches–which often focus on either rank selection or initialization in isolation–but also unifies both aspects within a single framework, enabling more effective and efficient adaptation. Extensive experiments across various architectures and modalities show that GoRA consistently outperforms existing LoRA-based methods while preserving the efficiency of vanilla LoRA. For example, when fine-tuning Llama3.1-8B-Base for mathematical reasoning, GoRA achieves a 5.13-point improvement over standard LoRA and even outperforms full fine-tuning by 2.05 points under high-rank settings. Code is available at: https://github.com/hhnqqq/MyTransformers.

Method: The authors theoretically revisit GNN message passing and identify a smoothness-generalization dilemma. They propose IGNN based on three design principles that enable distinct hop-wise generalization alongside improved overall generalization with adaptive smoothness.

Result: Benchmarking against 30 baselines demonstrates IGNN’s superiority and reveals notable universality in certain homophilic GNN variants. The model effectively addresses learning challenges in high-order homophilic neighborhoods and all heterophilic scenarios.

Conclusion: IGNN successfully addresses the smoothness-generalization dilemma in GNNs, providing a universal solution that works well across varying homophily levels through its innovative design principles and adaptive approach to generalization.

Abstract: Graph Neural Networks (GNNs) have achieved great success but are often considered to be challenged by varying levels of homophily in graphs. Recent \textit{empirical} studies have surprisingly shown that homophilic GNNs can perform well across datasets of different homophily levels with proper hyperparameter tuning, but the underlying theory and effective architectures remain unclear. To advance GNN universality across varying homophily, we theoretically revisit GNN message passing and uncover a novel \textit{smoothness-generalization dilemma}, where increasing hops inevitably enhances smoothness at the cost of generalization. This dilemma hinders learning in high-order homophilic neighborhoods and all heterophilic ones, where generalization is critical due to complex neighborhood class distributions that are sensitive to shifts induced by noise or sparsity. To address this, we introduce the Inceptive Graph Neural Network (IGNN) built on three simple yet effective design principles, which alleviate the dilemma by enabling distinct hop-wise generalization alongside improved overall generalization with adaptive smoothness. Benchmarking against 30 baselines demonstrates IGNN’s superiority and reveals notable universality in certain homophilic GNN variants. Our code and datasets are available at \href{https://github.com/galogm/IGNN}{https://github.com/galogm/IGNN}.

Method: Applied RLVR (reinforcement learning with verifiable reward) using only 1-2 training examples, tested across various models (Qwen2.5-Math, Llama3.2, DeepSeek) and RL algorithms (GRPO, PPO) with exploration promotion via entropy loss.

Result: 1-shot RLVR improved MATH500 performance from 36.0% to 73.6% (8.6% non-format gain) and average performance across six benchmarks from 17.6% to 35.7% (7.0% non-format gain), matching performance using 1.2k examples.

Conclusion: 1-shot RLVR is highly effective for improving math reasoning in LLMs, demonstrating cross-category generalization, post-saturation generalization, and highlighting the importance of policy gradient loss and exploration promotion.

Abstract: We show that reinforcement learning with verifiable reward using one training example (1-shot RLVR) is effective in incentivizing the math reasoning capabilities of large language models (LLMs). Applying RLVR to the base model Qwen2.5-Math-1.5B, we identify a single example that elevates model performance on MATH500 from 36.0% to 73.6% (8.6% improvement beyond format correction), and improves the average performance across six common mathematical reasoning benchmarks from 17.6% to 35.7% (7.0% non-format gain). This result matches the performance obtained using the 1.2k DeepScaleR subset (MATH500: 73.6%, average: 35.9%), which contains the aforementioned example. Furthermore, RLVR with only two examples even slightly exceeds these results (MATH500: 74.8%, average: 36.6%). Similar substantial improvements are observed across various models (Qwen2.5-Math-7B, Llama3.2-3B-Instruct, DeepSeek-R1-Distill-Qwen-1.5B), RL algorithms (GRPO and PPO), and different math examples. In addition, we identify some interesting phenomena during 1-shot RLVR, including cross-category generalization, increased frequency of self-reflection, and sustained test performance improvement even after the training accuracy has saturated, a phenomenon we term post-saturation generalization. Moreover, we verify that the effectiveness of 1-shot RLVR primarily arises from the policy gradient loss, distinguishing it from the “grokking” phenomenon. We also show the critical role of promoting exploration (e.g., by incorporating entropy loss with an appropriate coefficient) in 1-shot RLVR training. We also further discuss related observations about format correction, label robustness and prompt modification. These findings can inspire future work on RLVR efficiency and encourage a re-examination of recent progress and the underlying mechanisms in RLVR. All resources are open source at https://github.com/ypwang61/One-Shot-RLVR.

Method: Developed two novel distributionally robust DPO algorithms: Wasserstein DPO (WDPO) and Kullback-Leibler DPO (KLDPO), with sample complexity analysis and scalable gradient descent-style learning algorithms using approximations for the minimax loss functions.

Result: Empirical experiments using benchmark datasets and LLMs demonstrate superior performance of WDPO and KLDPO in substantially improving alignment when there is preference distribution shift.

Conclusion: Distributionally robust optimization provides an effective framework for addressing preference distribution shift in LLM alignment, with WDPO and KLDPO algorithms showing significant improvements over standard approaches.

Abstract: A major challenge in aligning large language models (LLMs) with human preferences is the issue of distribution shift. LLM alignment algorithms rely on static preference datasets, assuming that they accurately represent real-world user preferences. However, user preferences vary significantly across geographical regions, demographics, linguistic patterns, and evolving cultural trends. This preference distribution shift leads to catastrophic alignment failures in many real-world applications. We address this problem using the principled framework of distributionally robust optimization, and develop two novel distributionally robust direct preference optimization (DPO) algorithms, namely, Wasserstein DPO (WDPO) and Kullback-Leibler DPO (KLDPO). We characterize the sample complexity of learning the optimal policy parameters for WDPO and KLDPO. Moreover, we propose scalable gradient descent-style learning algorithms by developing suitable approximations for the challenging minimax loss functions of WDPO and KLDPO. Our empirical experiments using benchmark data sets and LLMs demonstrate the superior performance of WDPO and KLDPO in substantially improving the alignment when there is a preference distribution shift.

Method: ProxySPEX first fits gradient boosted trees to masked LLM outputs, then extracts important hierarchical interactions. It exploits the observation that higher-order interactions are accompanied by their lower-order subsets.

Result: ProxySPEX more faithfully reconstructs LLM outputs by 20% over marginal attribution approaches while using 10x fewer inferences than SPEX. It efficiently identifies influential features and provides scalable Shapley value approximations.

Conclusion: ProxySPEX enables efficient interaction attribution for LLMs, successfully applied to data attribution and mechanistic interpretability tasks, uncovering interactions between training samples and attention heads.

Abstract: Large Language Models (LLMs) have achieved remarkable performance by capturing complex interactions between input features. To identify these interactions, most existing approaches require enumerating all possible combinations of features up to a given order, causing them to scale poorly with the number of inputs $n$. Recently, Kang et al. (2025) proposed SPEX, an information-theoretic approach that uses interaction sparsity to scale to $n \approx 10^3$ features. SPEX greatly improves upon prior methods but requires tens of thousands of model inferences, which can be prohibitive for large models. In this paper, we observe that LLM feature interactions are often hierarchical – higher-order interactions are accompanied by their lower-order subsets – which enables more efficient discovery. To exploit this hierarchy, we propose ProxySPEX, an interaction attribution algorithm that first fits gradient boosted trees to masked LLM outputs and then extracts the important interactions. Experiments across four challenging high-dimensional datasets show that ProxySPEX more faithfully reconstructs LLM outputs by 20% over marginal attribution approaches while using $10\times$ fewer inferences than SPEX. By accounting for interactions, ProxySPEX efficiently identifies the most influential features, providing a scalable approximation of their Shapley values. Further, we apply ProxySPEX to two interpretability tasks. Data attribution, where we identify interactions among CIFAR-10 training samples that influence test predictions, and mechanistic interpretability, where we uncover interactions between attention heads, both within and across layers, on a question-answering task.

Method: Developed new model-free and model-based selectors with theoretical guarantees, and created a new experimental protocol that allows stable generation and better control of candidate value functions in an optimization-free manner.

Result: The new model-free selector, LSTD-Tournament, demonstrates promising empirical performance on Gym-Hopper.

Conclusion: The paper provides novel methods for hyperparameter tuning in OPE with both theoretical foundations and practical experimental protocols, showing promising results with the LSTD-Tournament selector.

Abstract: Holdout validation and hyperparameter tuning from data is a long-standing problem in offline reinforcement learning (RL). A standard framework is to use off-policy evaluation (OPE) methods to evaluate and select the policies, but OPE either incurs exponential variance (e.g., importance sampling) or has hyperparameters on their own (e.g., FQE and model-based). We focus on hyperparameter tuning for OPE itself, which is even more under-investigated. Concretely, we select among candidate value functions (“model-free”) or dynamics (“model-based”) to best assess the performance of a target policy. Concretely, we select among candidate value functions (model-free'') or dynamics models (model-based’’) to best assess the performance of a target policy. We develop: (1) new model-free and model-based selectors with theoretical guarantees, and (2) a new experimental protocol for empirically evaluating them. Compared to the model-free protocol in prior works, our new protocol allows for more stable generation and better control of candidate value functions in an optimization-free manner, and evaluation of model-free and model-based methods alike. We exemplify the protocol on Gym-Hopper, and find that our new model-free selector, LSTD-Tournament, demonstrates promising empirical performance.

Method: Theoretical and empirical analysis of bilinear operations involving multiplicative interactions between hidden units and input embeddings, examining their hierarchical structure for state tracking tasks.

Result: Bilinear operations form a natural inductive bias for hidden state evolution in state tracking tasks and create a hierarchy of complexity, with linear recurrent networks like Mamba at the lowest complexity level.

Conclusion: Bilinear state updates provide a fundamental framework for understanding hidden unit computation in RNNs, revealing a natural hierarchy of complexity in state tracking tasks.

Abstract: The role of hidden units in recurrent neural networks is typically seen as modeling memory, with research focusing on enhancing information retention through gating mechanisms. A less explored perspective views hidden units as active participants in the computation performed by the network, rather than passive memory stores. In this work, we revisit bilinear operations, which involve multiplicative interactions between hidden units and input embeddings. We demonstrate theoretically and empirically that they constitute a natural inductive bias for representing the evolution of hidden states in state tracking tasks. These are the simplest type of tasks that require hidden units to actively contribute to the behavior of the network. We also show that bilinear state updates form a natural hierarchy corresponding to state tracking tasks of increasing complexity, with popular linear recurrent networks such as Mamba residing at the lowest-complexity center of that hierarchy.

Method: Developed a metric based on Fréchet Distance estimated between datasets in a learned feature space, evaluating generation quality from a distributional perspective.

Result: Empirical results showed superiority of the proposed metric across different timescales and models, improving reliability of data-driven decision-making.

Conclusion: The Fréchet Distance-based metric effectively assesses synthetic data quality in smart grids, overcoming limitations of traditional evaluation methods and enhancing operational reliability.

Abstract: Generative artificial intelligence (AI) models in smart grids have advanced significantly in recent years due to their ability to generate large amounts of synthetic data, which would otherwise be difficult to obtain in the real world due to confidentiality constraints. A key challenge in utilizing such synthetic data is how to assess the data quality produced from such generative models. Traditional Euclidean distance-based metrics only reflect pair-wise relations between two individual samples, and could fail in evaluating quality differences between groups of synthetic datasets. In this work, we propose a novel metric based on the Fr'{e}chet Distance (FD) estimated between two datasets in a learned feature space. The proposed method evaluates the quality of generation from a distributional perspective. Empirical results demonstrate the superiority of the proposed metric across timescales and models, enhancing the reliability of data-driven decision-making in smart grid operations.

Method: 1) Adaptive difficulty-based online data selection using attention-based framework to prioritize moderately difficult questions; 2) Rollout replay mechanism to reuse recent rollouts and reduce computation.

Result: Reduces RL fine-tuning time by 23% to 62% across 6 LLM-dataset combinations while achieving same performance as original GRPO algorithm.

Conclusion: The proposed techniques significantly improve data efficiency in LLM RL fine-tuning, making the process more resource-effective without sacrificing performance.

Abstract: Reinforcement learning (RL) has become an effective approach for fine-tuning large language models (LLMs), particularly to enhance their reasoning capabilities. However, RL fine-tuning remains highly resource-intensive, and existing work has largely overlooked the problem of data efficiency. In this paper, we propose two techniques to improve data efficiency in LLM RL fine-tuning: difficulty-targeted online data selection and rollout replay. We introduce the notion of adaptive difficulty to guide online data selection, prioritizing questions of moderate difficulty that are more likely to yield informative learning signals. To estimate adaptive difficulty efficiently, we develop an attention-based framework that requires rollouts for only a small reference set of questions. The adaptive difficulty of the remaining questions is then estimated based on their similarity to this set. To further reduce rollout cost, we introduce a rollout replay mechanism inspired by experience replay in traditional RL. This technique reuses recent rollouts, lowering per-step computation while maintaining stable updates. Experiments across 6 LLM-dataset combinations show that our method reduces RL fine-tuning time by 23% to 62% while reaching the same level of performance as the original GRPO algorithm. Our code is available at https://github.com/ASTRAL-Group/data-efficient-llm-rl.

Method: Proposes ReDit method that dithers discrete reward signals by adding simple random noise, providing continuous exploratory gradients and introducing stochasticity in flat reward regions.

Result: ReDit achieves performance comparable to vanilla GRPO with only ~10% training steps, and shows 4% performance improvement when trained for similar duration. Visualizations confirm gradient issue mitigation.

Conclusion: ReDit effectively addresses gradient problems in discrete reward systems, enabling faster convergence and better performance through reward dithering with theoretical validation.

Abstract: DeepSeek-R1 has successfully enhanced Large Language Model (LLM) reasoning capabilities through its rule-based reward system. While it’s a ‘‘perfect’’ reward system that effectively mitigates reward hacking, such reward functions are often discrete. Our experimental observations suggest that discrete rewards can lead to gradient anomaly, unstable optimization, and slow convergence. To address this issue, we propose ReDit (Reward Dithering), a method that dithers the discrete reward signal by adding simple random noise. With this perturbed reward, exploratory gradients are continuously provided throughout the learning process, enabling smoother gradient updates and accelerating convergence. The injected noise also introduces stochasticity into flat reward regions, encouraging the model to explore novel policies and escape local optima. Experiments across diverse tasks demonstrate the effectiveness and efficiency of ReDit. On average, ReDit achieves performance comparable to vanilla GRPO with only approximately 10% the training steps, and furthermore, still exhibits a 4% performance improvement over vanilla GRPO when trained for a similar duration. Visualizations confirm significant mitigation of gradient issues with ReDit. Moreover, theoretical analyses are provided to further validate these advantages.

Method: Two-stage optimization: sharpness-aware evolutionary strategy stage for global exploration and loss landscape smoothing, followed by fine-grained local search via sparse zeroth-order optimization, using only forward passes.

Result: Significantly improves accuracy and convergence speed, achieving up to 7% average gain over state-of-the-art forward-only methods on CLIP models.

Conclusion: SharpZO provides an effective BP-free fine-tuning approach for vision language models that works well on memory-constrained devices while maintaining high performance.

Abstract: Fine-tuning vision language models (VLMs) has achieved remarkable performance across various downstream tasks; yet, it requires access to model gradients through backpropagation (BP), making them unsuitable for memory-constrained, inference-only edge devices. To address this limitation, previous work has explored various BP-free fine-tuning methods. However, these approaches often rely on high-variance evolutionary strategies (ES) or zeroth-order (ZO) optimization, and often fail to achieve satisfactory performance. In this paper, we propose a hybrid Sharpness-aware Zeroth-order optimization (SharpZO) approach, specifically designed to enhance the performance of ZO VLM fine-tuning via a sharpness-aware warm-up training. SharpZO features a two-stage optimization process: a sharpness-aware ES stage that globally explores and smooths the loss landscape to construct a strong initialization, followed by a fine-grained local search via sparse ZO optimization. The entire optimization relies solely on forward passes. Detailed theoretical analysis and extensive experiments on CLIP models demonstrate that SharpZO significantly improves accuracy and convergence speed, achieving up to 7% average gain over state-of-the-art forward-only methods.

Method: The authors analyze the estimator of sample quantiles from m-out-of-n resampling, proving a central limit theorem under mild moment conditions, showing the tightness of assumptions via counter-examples, and deriving Edgeworth expansions with exact convergence rates.

Result: The paper establishes parameter-free asymptotic distributions for practical statistics including quantiles for random walk Metropolis-Hastings and rewards of ergodic Markov decision processes, demonstrating the theory’s usefulness in modern estimation tasks.

Conclusion: The research provides foundational theoretical guarantees for m-out-of-n bootstrap in quantile estimation, with practical applications in modern statistical learning and estimation problems.

Abstract: The m-out-of-n bootstrap, originally proposed by Bickel, Gotze, and Zwet (1992), approximates the distribution of a statistic by repeatedly drawing m subsamples (with m much smaller than n) without replacement from an original sample of size n. It is now routinely used for robust inference with heavy-tailed data, bandwidth selection, and other large-sample applications. Despite its broad applicability across econometrics, biostatistics, and machine learning, rigorous parameter-free guarantees for the soundness of the m-out-of-n bootstrap when estimating sample quantiles have remained elusive. This paper establishes such guarantees by analyzing the estimator of sample quantiles obtained from m-out-of-n resampling of a dataset of size n. We first prove a central limit theorem for a fully data-driven version of the estimator that holds under a mild moment condition and involves no unknown nuisance parameters. We then show that the moment assumption is essentially tight by constructing a counter-example in which the CLT fails. Strengthening the assumptions slightly, we derive an Edgeworth expansion that provides exact convergence rates and, as a corollary, a Berry Esseen bound on the bootstrap approximation error. Finally, we illustrate the scope of our results by deriving parameter-free asymptotic distributions for practical statistics, including the quantiles for random walk Metropolis-Hastings and the rewards of ergodic Markov decision processes, thereby demonstrating the usefulness of our theory in modern estimation and learning tasks.

Method: DynaMix uses a multivariate ALRNN-based mixture-of-experts architecture that is pre-trained for dynamical system reconstruction. It can perform zero-shot inference on novel systems just from a provided context signal without any re-training.

Result: DynaMix faithfully forecasts long-term evolution of novel dynamical systems where existing time series foundation models fail, using only 0.1% of parameters and achieving orders of magnitude faster inference times. It outperforms TS foundation models in long-term statistics and often in short-term forecasts, even on real-world data not part of its training corpus.

Conclusion: Models built on dynamical system principles have huge potential for advancing the time series prediction field, as demonstrated by DynaMix’s superior performance compared to traditional time series models on DSR problems.

Abstract: Complex, temporally evolving phenomena, from climate to brain activity, are governed by dynamical systems (DS). DS reconstruction (DSR) seeks to infer generative surrogate models of these from observed data, reproducing their long-term behavior. Existing DSR approaches require purpose-training for any new system observed, lacking the zero-shot and in-context inference capabilities known from LLMs. Here we introduce DynaMix, a novel multivariate ALRNN-based mixture-of-experts architecture pre-trained for DSR, the first DSR model able to generalize zero-shot to out-of-domain DS. Just from a provided context signal, without any re-training, DynaMix faithfully forecasts the long-term evolution of novel DS where existing time series (TS) foundation models, like Chronos, fail – at a fraction of the number of parameters (0.1%) and orders of magnitude faster inference times. DynaMix outperforms TS foundation models in terms of long-term statistics, and often also short-term forecasts, even on real-world time series, like traffic or weather data, typically used for training and evaluating TS models, but not at all part of DynaMix’ training corpus. We illustrate some of the failure modes of TS models for DSR problems, and conclude that models built on DS principles may bear a huge potential also for advancing the TS prediction field.

Method: Developed a rigorous evaluation framework using automated dataset creation with masking strategy, creating a hallucination evaluation dataset from S&P 500 annual reports, and evaluating state-of-the-art LLMs on financial tabular data.

Result: Comprehensive evaluation of intrinsic hallucination patterns in LLMs on financial tabular data, providing insights into model reliability for financial applications.

Conclusion: The work provides a robust methodology for in-house LLM evaluation and serves as a critical step toward building more trustworthy and reliable financial Generative AI systems.

Abstract: Hallucination remains a critical challenge for deploying Large Language Models (LLMs) in finance. Accurate extraction and precise calculation from tabular data are essential for reliable financial analysis, since even minor numerical errors can undermine decision-making and regulatory compliance. Financial applications have unique requirements, often relying on context-dependent, numerical, and proprietary tabular data that existing hallucination benchmarks rarely capture. In this study, we develop a rigorous and scalable framework for evaluating intrinsic hallucinations in financial LLMs, conceptualized as a context-aware masked span prediction task over real-world financial documents. Our main contributions are: (1) a novel, automated dataset creation paradigm using a masking strategy; (2) a new hallucination evaluation dataset derived from S&P 500 annual reports; and (3) a comprehensive evaluation of intrinsic hallucination patterns in state-of-the-art LLMs on financial tabular data. Our work provides a robust methodology for in-house LLM evaluation and serves as a critical step toward building more trustworthy and reliable financial Generative AI systems.

Method: Uses continuous-time formulations from optimal control theory to derive insights for training and architecture design, creating a plug-and-play framework that integrates with existing Transformer models.

Result: Significant performance improvements across seven experiments including text generation, sentiment analysis, and image classification. Achieved 46% test loss reduction with nanoGPT using 42% fewer parameters, and 9.3% reduction with GPT-2.

Conclusion: This work establishes a new foundation for systematic Transformer improvements using optimal control theory, offering both performance gains and parameter efficiency with theoretical guarantees for generalization and robustness.

Abstract: We study Transformers through the perspective of optimal control theory, using tools from continuous-time formulations to derive actionable insights into training and architecture design. This framework improves the performance of existing Transformer models while providing desirable theoretical guarantees, including generalization and robustness. Our framework is designed to be plug-and-play, enabling seamless integration with established Transformer models and requiring only slight changes to the implementation. We conduct seven extensive experiments on tasks motivated by text generation, sentiment analysis, image classification, and point cloud classification. Experimental results show that the framework improves the test performance of the baselines, while being more parameter-efficient. On character-level text generation with nanoGPT, our framework achieves a 46% reduction in final test loss while using 42% fewer parameters. On GPT-2, our framework achieves a 9.3% reduction in final test loss, demonstrating scalability to larger models. To the best of our knowledge, this is the first work that applies optimal control theory to both the training and architecture of Transformers. It offers a new foundation for systematic, theory-driven improvements and moves beyond costly trial-and-error approaches.

Method: Systematic review through rigorous reproductions and isolated evaluations within a unified open-source framework, analyzing internal mechanisms, scenarios, and principles using fine-grained experiments across varying datasets, model sizes, and architectures.

Result: The study reveals that a minimalist combination of two techniques can unlock critic-free policy learning using vanilla PPO loss, consistently improving performance and surpassing strategies like GRPO and DAPO.

Conclusion: The paper provides clear guidelines for selecting RL techniques and a reliable roadmap for practitioners, demonstrating that simple, well-designed combinations can achieve superior results in RL for LLM reasoning.

Abstract: Reinforcement learning for LLM reasoning has rapidly emerged as a prominent research area, marked by a significant surge in related studies on both algorithmic innovations and practical applications. Despite this progress, several critical challenges remain, including the absence of standardized guidelines for employing RL techniques and a fragmented understanding of their underlying mechanisms. Additionally, inconsistent experimental settings, variations in training data, and differences in model initialization have led to conflicting conclusions, obscuring the key characteristics of these techniques and creating confusion among practitioners when selecting appropriate techniques. This paper systematically reviews widely adopted RL techniques through rigorous reproductions and isolated evaluations within a unified open-source framework. We analyze the internal mechanisms, applicable scenarios, and core principles of each technique through fine-grained experiments, including datasets of varying difficulty, model sizes, and architectures. Based on these insights, we present clear guidelines for selecting RL techniques tailored to specific setups, and provide a reliable roadmap for practitioners navigating the RL for the LLM domain. Finally, we reveal that a minimalist combination of two techniques can unlock the learning capability of critic-free policies using vanilla PPO loss. The results demonstrate that our simple combination consistently improves performance, surpassing strategies like GRPO and DAPO.

Method: Constructed LCDB 1.1, a large-scale database with high-resolution learning curves including modern learners (CatBoost, TabNet, RealMLP, TabPFN), and used statistically rigorous methods to analyze learning curve behavior across different algorithms and feature scalings.

Result: Found significant ill-behavior in approximately 15% of learning curves, almost twice as much as previous estimates. Identified specific learners that are more ill-behaved than others, and showed that different feature scalings rarely resolve ill-behavior.

Conclusion: Ill-behaved learning curves pose significant challenges for downstream tasks like learning curve fitting and model selection, highlighting the relevance of LCDB 1.1 as a challenging benchmark for future research on learning curve analysis.

Abstract: Sample-wise learning curves plot performance versus training set size. They are useful for studying scaling laws and speeding up hyperparameter tuning and model selection. Learning curves are often assumed to be well-behaved: monotone (i.e. improving with more data) and convex. By constructing the Learning Curves Database 1.1 (LCDB 1.1), a large-scale database with high-resolution learning curves including more modern learners (CatBoost, TabNet, RealMLP and TabPFN), we show that learning curves are less often well-behaved than previously thought. Using statistically rigorous methods, we observe significant ill-behavior in approximately 15% of the learning curves, almost twice as much as in previous estimates. We also identify which learners are to blame and show that specific learners are more ill-behaved than others. Additionally, we demonstrate that different feature scalings rarely resolve ill-behavior. We evaluate the impact of ill-behavior on downstream tasks, such as learning curve fitting and model selection, and find it poses significant challenges, underscoring the relevance and potential of LCDB 1.1 as a challenging benchmark for future research.

Method: Uses a single neural field with unified shared backbone conditioned on signal-specific latent variables (point clouds in Lie groups), integrated with Physics-Informed Neural Networks (PINNs) for accurate Eikonal travel-time modeling.

Result: Demonstrates superior performance, scalability, adaptability, and user controllability compared to existing Neural Operator-based Eikonal solver methods in seismic travel-time modeling on 2D, 3D, and spherical benchmark datasets.

Conclusion: The framework successfully integrates equivariant neural fields with neural eikonal solvers, providing an effective approach for modeling Eikonal solutions with enhanced efficiency, geometric properties, and steerability across diverse manifolds.

Abstract: We introduce Equivariant Neural Eikonal Solvers, a novel framework that integrates Equivariant Neural Fields (ENFs) with Neural Eikonal Solvers. Our approach employs a single neural field where a unified shared backbone is conditioned on signal-specific latent variables - represented as point clouds in a Lie group - to model diverse Eikonal solutions. The ENF integration ensures equivariant mapping from these latent representations to the solution field, delivering three key benefits: enhanced representation efficiency through weight-sharing, robust geometric grounding, and solution steerability. This steerability allows transformations applied to the latent point cloud to induce predictable, geometrically meaningful modifications in the resulting Eikonal solution. By coupling these steerable representations with Physics-Informed Neural Networks (PINNs), our framework accurately models Eikonal travel-time solutions while generalizing to arbitrary Riemannian manifolds with regular group actions. This includes homogeneous spaces such as Euclidean, position-orientation, spherical, and hyperbolic manifolds. We validate our approach through applications in seismic travel-time modeling of 2D, 3D, and spherical benchmark datasets. Experimental results demonstrate superior performance, scalability, adaptability, and user controllability compared to existing Neural Operator-based Eikonal solver methods.

Method: Identified the bottleneck between encoder output and dense layers, and proposed using global average pooling as a simple solution instead of complex architectural changes.

Result: Global average pooling successfully targets the bottleneck and improves scaling capabilities without the complexity of previous approaches.

Conclusion: The bottleneck between encoder and dense layers is the main limiting factor in scaling deep RL, and global average pooling provides an effective, simple solution.

Abstract: Scaling deep reinforcement learning in pixel-based environments presents a significant challenge, often resulting in diminished performance. While recent works have proposed algorithmic and architectural approaches to address this, the underlying cause of the performance drop remains unclear. In this paper, we identify the connection between the output of the encoder (a stack of convolutional layers) and the ensuing dense layers as the main underlying factor limiting scaling capabilities; we denote this connection as the bottleneck, and we demonstrate that previous approaches implicitly target this bottleneck. As a result of our analyses, we present global average pooling as a simple yet effective way of targeting the bottleneck, thereby avoiding the complexity of earlier approaches.

Method: Developed FITS prototype using transformer-based system with fine-tuned BERT models (including scientific and climate-specific pretrained models) on curated SustainableTextileCorpus, optimized via Bayesian optimization, with interactive interface for data search and analysis.

Result: Successfully created FITS system that extracts sustainability information from NGO reports and scientific publications. Evaluated through user focus groups showing positive feedback on usability, design, and content clarity. Provided SustainableTextileCorpus dataset for future research.

Conclusion: Domain-adapted NLP like FITS can effectively promote informed decision-making in sustainability and demonstrates broader potential of AI for climate-related challenges. The work provides valuable dataset and methodology for future updates.

Abstract: Access to credible sustainability information in the fashion industry remains limited and challenging to interpret, despite growing public and regulatory demands for transparency. General-purpose language models often lack domain-specific knowledge and tend to “hallucinate”, which is particularly harmful for fields where factual correctness is crucial. This work explores how Natural Language Processing (NLP) techniques can be applied to classify sustainability data for fashion brands, thereby addressing the scarcity of credible and accessible information in this domain. We present a prototype Fashion Information Tool for Sustainability (FITS), a transformer-based system that extracts and classifies sustainability information from credible, unstructured text sources: NGO reports and scientific publications. Several BERT-based language models, including models pretrained on scientific and climate-specific data, are fine-tuned on our curated corpus using a domain-specific classification schema, with hyperparameters optimized via Bayesian optimization. FITS allows users to search for relevant data, analyze their own data, and explore the information via an interactive interface. We evaluated FITS in two focus groups of potential users concerning usability, visual design, content clarity, possible use cases, and desired features. Our results highlight the value of domain-adapted NLP in promoting informed decision-making and emphasize the broader potential of AI applications in addressing climate-related challenges. Finally, this work provides a valuable dataset, the SustainableTextileCorpus, along with a methodology for future updates. Code available at github(.)com/daphne12345/FITS.

Method: Proposed Hölder-DPO, a principled alignment loss with provable redescending property that estimates clean data distribution from noisy feedback and provides gradient-free mislabel detection.

Result: Achieves state-of-the-art robust alignment performance, accurately detects mislabels in controlled datasets, and reveals substantial noise in Anthropic HH-RLHF dataset where removing mislabels improves alignment performance.

Conclusion: Hölder-DPO provides a theoretically grounded solution for robust alignment under noisy human feedback and enables scalable automated dataset valuation without manual verification.

Abstract: Despite the importance of aligning language models with human preferences, crowd-sourced human feedback is often noisy – for example, preferring less desirable responses – posing a fundamental challenge to alignment. A truly robust alignment objective should yield identical model parameters even under severe label noise, a property known as redescending. We prove that no existing alignment methods satisfy this property. To address this, we propose H"older-DPO, the first principled alignment loss with a provable redescending property, enabling estimation of the clean data distribution from noisy feedback. The aligned model estimates the likelihood of clean data, providing a theoretically grounded metric for dataset valuation that identifies the location and fraction of mislabels. This metric is gradient-free, enabling scalable and automated human feedback valuation without costly manual verification or clean validation dataset. H"older-DPO achieves state-of-the-art robust alignment performance while accurately detecting mislabels in controlled datasets. Finally, applied to Anthropic HH-RLHF dataset, it reveals substantial noise levels and removing these mislabels significantly improves alignment performance across methods. The code is available at https://github.com/ma921/HolderDPO.

Method: Developed KnotGym environment with goal-oriented rope manipulation tasks scaled by knot crossing complexity, evaluated model-based RL, model-predictive control, and chain-of-thought reasoning approaches.

Result: KnotGym successfully highlights core challenges in integrating perception, spatial reasoning, and manipulation, providing a testbed for evaluating different AI methods on complex spatial tasks.

Conclusion: KnotGym serves as an effective environment for benchmarking spatial reasoning and manipulation capabilities, demonstrating clear challenges that current methods face in complex knot manipulation tasks.

Abstract: We propose KnotGym, an interactive environment for complex, spatial reasoning and manipulation. KnotGym includes goal-oriented rope manipulation tasks with varying levels of complexity, all requiring acting from pure image observations. Tasks are defined along a clear and quantifiable axis of complexity based on the number of knot crossings, creating a natural generalization test. KnotGym has a simple observation space, allowing for scalable development, yet it highlights core challenges in integrating acute perception, spatial reasoning, and grounded manipulation. We evaluate methods of different classes, including model-based RL, model-predictive control, and chain-of-thought reasoning, and illustrate the challenges KnotGym presents. KnotGym is available at https://github.com/lil-lab/knotgym.

Method: Combines Variational Encoding of Noisy Inputs (VENI) for dimensionality reduction with Variational Identification of Nonlinear Dynamics (VINDy) to learn interpretable dynamics, enabling uncertainty quantification through Variational Inference (VICI).

Result: Successfully demonstrated on Roessler system with various noise levels and PDE benchmarks in structural mechanics and fluid dynamics.

Conclusion: The proposed VENI-VINDy-VICI framework provides interpretable, accurate ROMs with built-in uncertainty quantification for complex systems with unknown or partially known dynamics.

Abstract: The simulation of many complex phenomena in engineering and science requires solving expensive, high-dimensional systems of partial differential equations (PDEs). To circumvent this, reduced-order models (ROMs) have been developed to speed up computations. However, when governing equations are unknown or partially known, typically ROMs lack interpretability and reliability of the predicted solutions. In this work we present a data-driven, non-intrusive framework for building ROMs where the latent variables and dynamics are identified in an interpretable manner and uncertainty is quantified. Starting from a limited amount of high-dimensional, noisy data the proposed framework constructs an efficient ROM by leveraging variational autoencoders for dimensionality reduction along with a newly introduced, variational version of sparse identification of nonlinear dynamics (SINDy), which we refer to as Variational Identification of Nonlinear Dynamics (VINDy). In detail, the method consists of Variational Encoding of Noisy Inputs (VENI) to identify the distribution of reduced coordinates. Simultaneously, we learn the distribution of the coefficients of a pre-determined set of candidate functions by VINDy. Once trained offline, the identified model can be queried for new parameter instances and new initial conditions to compute the corresponding full-time solutions. The probabilistic setup enables uncertainty quantification as the online testing consists of Variational Inference naturally providing Certainty Intervals (VICI). In this work we showcase the effectiveness of the newly proposed VINDy method in identifying interpretable and accurate dynamical system for the Roessler system with different noise intensities and sources. Then the performance of the overall method - named VENI, VINDy, VICI - is tested on PDE benchmarks including structural mechanics and fluid dynamics.

Method: The authors provide formal analysis comparing CoT and Looped Transformers, examining their capabilities in simulating parallel computations and approximate inference.

Result: Looped Transformers efficiently simulate parallel computations for deterministic tasks (DAG evaluation), while CoT with stochastic decoding excels at approximate inference for compositional structures (self-reducible problems).

Conclusion: The analysis reveals task-specific strengths of each approach, providing practical guidance for choosing between reasoning paradigms based on whether depth-driven recursion or approximate inference is more suitable.

Abstract: Chain-of-Thought (CoT) and Looped Transformers have been shown to empirically improve performance on reasoning tasks and to theoretically enhance expressivity by recursively increasing the number of computational steps. However, their comparative capabilities are still not well understood. In this paper, we provide a formal analysis of their respective strengths and limitations. We show that Looped Transformers can efficiently simulate parallel computations for deterministic tasks, which we formalize as evaluation over directed acyclic graphs. In contrast, CoT with stochastic decoding excels at approximate inference for compositional structures, namely self-reducible problems. These separations suggest the tasks for which depth-driven recursion is more suitable, thereby offering practical cues for choosing between reasoning paradigms.

Method: 1) Introduced normalization in relative transformation for invariance to non-isotropic rescalings and permutations; 2) Applied topological densification during fine-tuning using a regularization loss that encourages clustering within classes.

Result: Both proposed variations showed improved performance on zero-shot model stitching in empirical investigation on natural language tasks.

Conclusion: The normalization procedure and topological densification effectively enhance relative representations for zero-shot model stitching, providing better handling of symmetries and improved class separation.

Abstract: Relative representations are an established approach to zero-shot model stitching, consisting of a non-trainable transformation of the latent space of a deep neural network. Based on insights of topological and geometric nature, we propose two improvements to relative representations. First, we introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations. The latter coincides with the symmetries in parameter space induced by common activation functions. Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes. We provide an empirical investigation on a natural language task, where both the proposed variations yield improved performance on zero-shot model stitching.

Method: RBO simulates a rigid sphere of finite radius rolling on the loss landscape, which generalizes Gradient Descent (GD) and simplifies to it in the infinitesimal limit. The radius hyperparameter controls the scale at which RBO interacts with the landscape, incorporating information from larger regions.

Result: RBO demonstrates promising results on MNIST and CIFAR-10/100 datasets, showing improvements in convergence speed, training accuracy, and generalization performance compared to SGD, SAM, and Entropy-SGD.

Conclusion: RBO effectively addresses the problem of noisy data in neural network optimization by leveraging large-scale landscape geometry, proving to have a smoothing effect on the loss function and achieving better generalization than existing methods.

Abstract: Training large neural networks (NNs) requires optimizing high-dimensional data-dependent loss functions. The optimization landscape of these functions is often highly complex and textured, even fractal-like, with many spurious local minima, ill-conditioned valleys, degenerate points, and saddle points. Complicating things further is the fact that these landscape characteristics are a function of the data, meaning that noise in the training data can propagate forward and give rise to unrepresentative small-scale geometry. This poses a difficulty for gradient-based optimization methods, which rely on local geometry to compute updates and are, therefore, vulnerable to being derailed by noisy data. In practice,this translates to a strong dependence of the optimization dynamics on the noise in the data, i.e., poor generalization performance. To remediate this problem, we propose a new optimization procedure: Rolling Ball Optimizer (RBO), that breaks this spatial locality by incorporating information from a larger region of the loss landscape in its updates. We achieve this by simulating the motion of a rigid sphere of finite radius rolling on the loss landscape, a straightforward generalization of Gradient Descent (GD) that simplifies into it in the infinitesimal limit. The radius serves as a hyperparameter that determines the scale at which RBO sees the loss landscape, allowing control over the granularity of its interaction therewith. We are motivated by the intuition that the large-scale geometry of the loss landscape is less data-specific than its fine-grained structure, and that it is easier to optimize. We support this intuition by proving that our algorithm has a smoothing effect on the loss function. Evaluation against SGD, SAM, and Entropy-SGD, on MNIST and CIFAR-10/100 demonstrates promising results in terms of convergence speed, training accuracy, and generalization performance.

Method: Applied ring attractors by building exogenous models and integrating them into DRL agents, mapping actions to ring locations and decoding based on neural activity.

Result: Achieved 53% performance improvement over selected baselines on the Atari 100k benchmark.

Conclusion: Ring attractors provide an effective biologically plausible mechanism to significantly enhance DRL performance through improved action space representation and neural organization.

Abstract: Ring attractors, mathematical models inspired by neural circuit dynamics, provide a biologically plausible mechanism to improve learning speed and accuracy in Reinforcement Learning (RL). Serving as specialized brain-inspired structures that encode spatial information and uncertainty, ring attractors explicitly encode the action space, facilitate the organization of neural activity, and enable the distribution of spatial representations across the neural network in the context of Deep Reinforcement Learning (DRL). These structures also provide temporal filtering that stabilizes action selection during exploration, for example, by preserving the continuity between rotation angles in robotic control or adjacency between tactical moves in game-like environments. The application of ring attractors in the action selection process involves mapping actions to specific locations on the ring and decoding the selected action based on neural activity. We investigate the application of ring attractors by both building an exogenous model and integrating them as part of DRL agents. Our approach significantly improves state-of-the-art performance on the Atari 100k benchmark, achieving a 53% increase in performance over selected baselines.

Method: Combines virtual queues and request satisfaction prediction to solve per-request optimization problems, learning model satisfaction probabilities through real-time user interactions.

Result: Achieves 2x cost savings compared to existing LLM routing techniques across various state-of-the-art LLM benchmarks.

Conclusion: MESS+ provides a practical solution for cost-efficient LLM routing with rigorous SLA guarantees, addressing the model selection challenge in open-weight LLM zoos.

Abstract: Open-weight large language model (LLM) zoos provide access to numerous high-quality models, but selecting the appropriate model for specific tasks remains challenging and requires technical expertise. Most users simply want factually correct, safe, and satisfying responses without concerning themselves with model technicalities, while inference service providers prioritize minimizing operating costs. These competing interests are typically mediated through service level agreements (SLAs) that guarantee minimum service quality. We introduce MESS+, a stochastic optimization algorithm for cost-optimal LLM request routing while providing rigorous SLA compliance guarantees. MESS+ learns request satisfaction probabilities of LLMs in real-time as users interact with the system, based on which model selection decisions are made by solving a per-request optimization problem. Our algorithm includes a novel combination of virtual queues and request satisfaction prediction, along with a theoretical analysis of cost optimality and constraint satisfaction. Across a wide range of state-of-the-art LLM benchmarks, MESS+ achieves an average of $2\times$ cost savings compared to existing LLM routing techniques.

Method: Propose a PAC-Bayesian framework that links adversarial robustness to posterior covariance and loss curvature. Analyze discrete-time SGD dynamics near local optimum under quadratic loss, deriving closed-form posterior covariances for stationary and non-stationary regimes.

Result: Reveals how learning rate, gradient noise, and Hessian structure jointly shape robust generalization during training. Empirically visualizes theoretical quantities to explain robust overfitting.

Conclusion: Fundamentally explains robust overfitting phenomenon and why flatness-promoting techniques like adversarial weight perturbation improve robustness.

Abstract: Despite significant progress in adversarially robust learning, the underlying mechanisms that govern robust generalization remain poorly understood. We propose a novel PAC-Bayesian framework that explicitly links adversarial robustness to the posterior covariance of model parameters and the curvature of the adversarial loss landscape. By characterizing discrete-time SGD dynamics near a local optimum under quadratic loss, we derive closed-form posterior covariances for both the stationary regime and the early phase of non-stationary transition. Our analyses reveal how key factors, such as learning rate, gradient noise, and Hessian structure, jointly shape robust generalization during training. Through empirical visualizations of these theoretical quantities, we fundamentally explain the phenomenon of robust overfitting and shed light on why flatness-promoting techniques like adversarial weight perturbation help to improve robustness.

Method: Used county and district level aggregations of EMS overdose data, evaluated models with different complexity levels, and tested various relevant covariates to determine their impact on forecasting performance.

Result: Useful predictions with limited error can be generated for different types of regions, and high performance can be achieved using commonly available covariates and relatively simple forecasting models.

Conclusion: Effective opioid overdose forecasting is feasible using accessible data and straightforward models, enabling practical resource allocation for public health agencies.

Abstract: We present efforts in the fields of machine learning and time series forecasting to accurately predict counts of future suspected opioid overdoses recorded by Emergency Medical Services (EMS) in the state of Kentucky. Forecasts help government agencies properly prepare and distribute resources related to opioid overdoses. Our approach uses county and district level aggregations of suspected opioid overdose encounters and forecasts future counts for different time intervals. Models with different levels of complexity were evaluated to minimize forecasting error. A variety of additional covariates relevant to opioid overdoses and public health were tested to determine their impact on model performance. Our evaluation shows that useful predictions can be generated with limited error for different types of regions, and high performance can be achieved using commonly available covariates and relatively simple forecasting models.

Method: The authors develop a theoretically justified data augmentation approach for QPs via MPNNs, generating diverse but optimality-preserving instances, and integrate these augmentations into a self-supervised contrastive learning framework for pretraining.

Result: Extensive experiments show that the proposed approach improves generalization in supervised scenarios and enables effective transfer learning to related optimization problems.

Conclusion: The principled data augmentation and contrastive learning framework enhances the performance and robustness of learning-to-optimize MPNNs for quadratic programs, particularly in data-scarce settings.

Abstract: Linear and quadratic optimization are crucial in numerous real-world applications, ranging from training machine learning models to solving integer linear programs. Recently, learning-to-optimize methods (L2O) for linear (LPs) or quadratic programs (QPs) using message-passing graph neural networks (MPNNs) have gained traction, promising lightweight, data-driven proxies for solving such optimization problems. For example, they replace the costly computation of strong branching scores in branch-and-bound solvers, thereby reducing the need to solve many such optimization problems. However, robust L2O MPNNs remain challenging in data-scarce settings, especially when addressing complex optimization problems such as QPs. This work introduces a principled approach to data augmentation tailored for QPs via MPNNs. Our method leverages theoretically justified data augmentation techniques to generate diverse yet optimality-preserving instances. Furthermore, we integrate these augmentations into a self-supervised contrastive learning framework, thereby pretraining MPNNs for improved performance on L2O tasks. Extensive experiments demonstrate that our approach improves generalization in supervised scenarios and facilitates effective transfer learning to related optimization problems.

Method: The proposed method tracks the impact of gradient updates on the objective for each timestep and adaptively selects timesteps that are most likely to minimize the objective effectively, using non-uniform sampling that prioritizes critical timesteps.

Result: Experimental results show the approach accelerates training process and leads to improved performance at convergence. The method demonstrates robust performance across various datasets, scheduling strategies, and diffusion architectures, outperforming previously proposed timestep sampling and weighting heuristics.

Conclusion: The non-uniform timestep sampling method effectively addresses the computational bottleneck in diffusion model training by adaptively focusing on critical timesteps, resulting in faster convergence and better performance across diverse settings.

Abstract: As a highly expressive generative model, diffusion models have demonstrated exceptional success across various domains, including image generation, natural language processing, and combinatorial optimization. However, as data distributions grow more complex, training these models to convergence becomes increasingly computationally intensive. While diffusion models are typically trained using uniform timestep sampling, our research shows that the variance in stochastic gradients varies significantly across timesteps, with high-variance timesteps becoming bottlenecks that hinder faster convergence. To address this issue, we introduce a non-uniform timestep sampling method that prioritizes these more critical timesteps. Our method tracks the impact of gradient updates on the objective for each timestep, adaptively selecting those most likely to minimize the objective effectively. Experimental results demonstrate that this approach not only accelerates the training process, but also leads to improved performance at convergence. Furthermore, our method shows robust performance across various datasets, scheduling strategies, and diffusion architectures, outperforming previously proposed timestep sampling and weighting heuristics that lack this degree of robustness.

Method: Word-level temporal alignment synchronizes audio embeddings with text tokens using transcription timestamps. Gated Cross-Attention Fusion allows audio features to attend over text representations. Prosodic cues are incorporated by inserting pause tokens and generating audio embeddings for silent intervals.

Result: Achieved 87.35% accuracy in Leave-One-Subject-Out setup and 90.36% in 5-fold Cross-Validation on ADReSSo dataset, outperforming state-of-the-art methods. Ablation studies confirmed benefits of alignment, attention fusion, and prosodic modeling.

Conclusion: Fine-grained multimodal alignment and fusion significantly improve Alzheimer’s detection. The approach enables more precise cross-modal interactions and provides interpretable insights through feature analysis and gradient attribution.

Abstract: Early detection of cognitive disorders such as Alzheimer’s disease is critical for enabling timely clinical intervention and improving patient outcomes. In this work, we introduce CogniAlign, a multimodal architecture for Alzheimer’s detection that integrates audio and textual modalities, two non-intrusive sources of information that offer complementary insights into cognitive health. Unlike prior approaches that fuse modalities at a coarse level, CogniAlign leverages a word-level temporal alignment strategy that synchronizes audio embeddings with corresponding textual tokens based on transcription timestamps. This alignment supports the development of token-level fusion techniques, enabling more precise cross-modal interactions. To fully exploit this alignment, we propose a Gated Cross-Attention Fusion mechanism, where audio features attend over textual representations, guided by the superior unimodal performance of the text modality. In addition, we incorporate prosodic cues, specifically interword pauses, by inserting pause tokens into the text and generating audio embeddings for silent intervals, further enriching both streams. We evaluate CogniAlign on the ADReSSo dataset, where it achieves an accuracy of 87.35% over a Leave-One-Subject-Out setup and of 90.36% over a 5 fold Cross-Validation, outperforming existing state-of-the-art methods. A detailed ablation study confirms the advantages of our alignment strategy, attention-based fusion, and prosodic modeling. Finally, we perform a corpus analysis to assess the impact of the proposed prosodic features and apply Integrated Gradients to identify the most influential input segments used by the model in predicting cognitive health outcomes.

Method: Developed new algorithms that learn from positive data alone, achieving optimal sample complexity in the realizable case and establishing multiplicative approximation guarantees in the agnostic case.

Result: The approach achieves optimal sample complexity in realizable settings and provides multiplicative approximation guarantees where additive regret is impossible, showing sharp statistical and algorithmic separations from standard settings.

Conclusion: The proposed framework successfully addresses learning under one-sided feedback, overcoming limitations of classical methods like Empirical Risk Minimization that provably fail in such settings.

Abstract: Precision and Recall are fundamental metrics in machine learning tasks where both accurate predictions and comprehensive coverage are essential, such as in multi-label learning, language generation, medical studies, and recommender systems. A key challenge in these settings is the prevalence of one-sided feedback, where only positive examples are observed during training–e.g., in multi-label tasks like tagging people in Facebook photos, we may observe only a few tagged individuals, without knowing who else appears in the image. To address learning under such partial feedback, we introduce a Probably Approximately Correct (PAC) framework in which hypotheses are set functions that map each input to a set of labels, extending beyond single-label predictions and generalizing classical binary, multi-class, and multi-label models. Our results reveal sharp statistical and algorithmic separations from standard settings: classical methods such as Empirical Risk Minimization provably fail, even for simple hypothesis classes. We develop new algorithms that learn from positive data alone, achieving optimal sample complexity in the realizable case, and establishing multiplicative–rather than additive-approximation guarantees in the agnostic case, where achieving additive regret is impossible.

Method: Integrates context-aware feature extraction with predictive network using stacked attention blocks, featuring a core Mixture-of-Time module that adaptively integrates predictions from multiple temporal contexts by learning variable-specific reliability.

Result: Demonstrates superior skill in predicting key ocean variables including temperature, salinity, and currents across multiple depths.

Conclusion: FuXi-Ocean represents a significant advancement in data-driven ocean forecasting, enabling high-resolution sub-daily predictions while addressing error accumulation challenges.

Abstract: Accurate, high-resolution ocean forecasting is crucial for maritime operations and environmental monitoring. While traditional numerical models are capable of producing sub-daily, eddy-resolving forecasts, they are computationally intensive and face challenges in maintaining accuracy at fine spatial and temporal scales. In contrast, recent data-driven approaches offer improved computational efficiency and emerging potential, yet typically operate at daily resolution and struggle with sub-daily predictions due to error accumulation over time. We introduce FuXi-Ocean, the first data-driven global ocean forecasting model achieving six-hourly predictions at eddy-resolving 1/12{\deg} spatial resolution, reaching depths of up to 1500 meters. The model architecture integrates a context-aware feature extraction module with a predictive network employing stacked attention blocks. The core innovation is the Mixture-of-Time (MoT) module, which adaptively integrates predictions from multiple temporal contexts by learning variable-specific reliability , mitigating cumulative errors in sequential forecasting. Through comprehensive experimental evaluation, FuXi-Ocean demonstrates superior skill in predicting key variables, including temperature, salinity, and currents, across multiple depths.

Method: Proposes a new graph representation capturing action information and uses GNN architecture with GRUs to learn locally consistent action rankings from small solved instances.

Result: Outperforms value-function and other action ranking baselines in success rate and plan quality, achieving better generalization to larger problems than training data.

Conclusion: Action ranking with GNNs provides an effective approach for learning relational policies that scale well to larger planning problems beyond training instances.

Abstract: We propose a novel approach to learn relational policies for classical planning based on learning to rank actions. We introduce a new graph representation that explicitly captures action information and propose a Graph Neural Network (GNN) architecture augmented with Gated Recurrent Units (GRUs) to learn action rankings. Unlike value-function based approaches that must learn a globally consistent function, our action ranking method only needs to learn locally consistent ranking. Our model is trained on data generated from small problem instances that are easily solved by planners and is applied to significantly larger instances where planning is computationally prohibitive. Experimental results across standard planning benchmarks demonstrate that our action-ranking approach not only achieves better generalization to larger problems than those used in training but also outperforms multiple baselines (value function and action ranking) methods in terms of success rate and plan quality.

Method: Propose Unlearn-Noise-Distill-on-Outputs (UNDO) - a scalable method that distills an unlearned model into a noised copy of itself, establishing a tunable tradeoff between compute cost and robustness.

Result: UNDO matches the robustness of retraining from scratch with perfect data filtering while using only 60-80% of the compute and requiring only 0.01% of pretraining data to be labeled. It also robustifies unlearning on the WMDP benchmark.

Conclusion: Distillation robustifies unlearning, and incorporating an unlearning step before distillation offers a convenient path to robust capability removal in practice.

Abstract: Current LLM unlearning methods are not robust. A few steps of finetuning can revert their effects. We begin by showing that this is true even for an idealized form of unlearning: training to imitate a model that was never trained on unwanted information. This shows that training a model can drastically modify its input-output behavior while leaving its underlying capabilities intact. In light of this dynamic, we show our main result. Training a randomly initialized student on the outputs of an unlearned model transfers behaviors while leaving latent capabilities behind. In short, distillation robustifies unlearning. Based on this result, we propose Unlearn-Noise-Distill-on-Outputs (UNDO), a scalable method that distills an unlearned model into a noised copy of itself. UNDO introduces a tunable tradeoff between compute cost and robustness, establishing a new Pareto frontier on synthetic language and arithmetic tasks. At its strongest setting, UNDO matches the robustness of a model retrained from scratch with perfect data filtering while using only 60-80% of the compute and requiring only 0.01% of the pretraining data to be labeled. We also show that UNDO robustifies unlearning on the more realistic Weapons of Mass Destruction Proxy (WMDP) benchmark. Since distillation is widely used in practice, incorporating an unlearning step beforehand offers a convenient path to robust capability removal.

Method: Uses a neural network ensemble as a utility surrogate model that naturally integrates monotonicity constraints and supports pairwise comparison data.

Result: The proposed method outperforms state-of-the-art approaches and shows robustness to noise in utility evaluations. An ablation study confirms monotonicity’s critical role in performance enhancement.

Conclusion: Incorporating monotonicity constraints in utility surrogate models significantly improves Bayesian Optimization with Preference Exploration, making the approach more effective and robust for multi-objective optimization problems.

Abstract: Many real-world black-box optimization problems have multiple conflicting objectives. Rather than attempting to approximate the entire set of Pareto-optimal solutions, interactive preference learning allows to focus the search on the most relevant subset. However, few previous studies have exploited the fact that utility functions are usually monotonic. In this paper, we address the Bayesian Optimization with Preference Exploration (BOPE) problem and propose using a neural network ensemble as a utility surrogate model. This approach naturally integrates monotonicity and supports pairwise comparison data. Our experiments demonstrate that the proposed method outperforms state-of-the-art approaches and exhibits robustness to noise in utility evaluations. An ablation study highlights the critical role of monotonicity in enhancing performance.

Method: The authors construct projection-based Lyapunov functions that certify the convergence of rewards and costs to an optimal region, even under full heterogeneity, and develop an efficiently computable policy.

Result: The proposed policy achieves an O(1/√N) optimality gap in the long-run average reward per arm for fully heterogeneous WCMDPs as N becomes large, which is the first asymptotic optimality result for such problems.

Conclusion: The paper successfully addresses the curse of dimensionality in fully heterogeneous WCMDPs through novel Lyapunov function construction and provides the first asymptotic optimality guarantee for this challenging class of problems.

Abstract: Heterogeneity poses a fundamental challenge for many real-world large-scale decision-making problems but remains largely understudied. In this paper, we study the fully heterogeneous setting of a prominent class of such problems, known as weakly-coupled Markov decision processes (WCMDPs). Each WCMDP consists of $N$ arms (or subproblems), which have distinct model parameters in the fully heterogeneous setting, leading to the curse of dimensionality when $N$ is large. We show that, under mild assumptions, an efficiently computable policy achieves an $O(1/\sqrt{N})$ optimality gap in the long-run average reward per arm for fully heterogeneous WCMDPs as $N$ becomes large. This is the first asymptotic optimality result for fully heterogeneous average-reward WCMDPs. Our main technical innovation is the construction of projection-based Lyapunov functions that certify the convergence of rewards and costs to an optimal region, even under full heterogeneity.

Method: Developed an interpretable climate model emulator based on causal representation learning with a novel Bayesian filter for stable long-term autoregressive emulation.

Result: The emulator learns accurate climate dynamics and demonstrates the importance of each component on realistic synthetic datasets and data from two widely deployed climate models.

Conclusion: The proposed causal representation learning approach with Bayesian filtering enables accurate and stable long-term climate model emulation while maintaining interpretability.

Abstract: Traditional models of climate change use complex systems of coupled equations to simulate physical processes across the Earth system. These simulations are highly computationally expensive, limiting our predictions of climate change and analyses of its causes and effects. Machine learning has the potential to quickly emulate data from climate models, but current approaches are not able to incorporate physically-based causal relationships. Here, we develop an interpretable climate model emulator based on causal representation learning. We derive a novel approach including a Bayesian filter for stable long-term autoregressive emulation. We demonstrate that our emulator learns accurate climate dynamics, and we show the importance of each one of its components on a realistic synthetic dataset and data from two widely deployed climate models.

Method: Proposes Deconfounded Empirical Risk Minimization with sufficient representation constraint to transfer validity across experiments, ensuring conditional calibration for valid PPCI.

Result: Method validated on synthetic and real-world data, solving problems impossible for standard Empirical Risk Minimization, achieving valid causal inference without human annotations using foundational models.

Conclusion: PPCI with the proposed method enables valid causal inference in unlabeled experiments, overcoming annotation cost barriers in scientific research.

Abstract: In many scientific experiments, the data annotating cost constraints the pace for testing novel hypotheses. Yet, modern machine learning pipelines offer a promising solution, provided their predictions yield correct conclusions. We focus on Prediction-Powered Causal Inferences (PPCI), i.e., estimating the treatment effect in an unlabeled target experiment, relying on training data with the same outcome annotated but potentially different treatment or effect modifiers. We first show that conditional calibration guarantees valid PPCI at population level. Then, we introduce a sufficient representation constraint transferring validity across experiments, which we propose to enforce in practice in Deconfounded Empirical Risk Minimization, our new model-agnostic training objective. We validate our method on synthetic and real-world scientific data, solving impossible problem instances for Empirical Risk Minimization even with standard invariance constraints. In particular, for the first time, we achieve valid causal inference on a scientific experiment with complex recording and no human annotations, fine-tuning a foundational model on our similar annotated experiment.

Method: Uses conditional diffusion models to generate synthetic bias-aligned images, creates a bias amplifier model, and incorporates it into end-to-end and two-step unsupervised debiasing approaches.

Result: Beats current state-of-the-art methods on multiple benchmark datasets by addressing bias-conflicting training samples memorization issue.

Conclusion: DDB demonstrates potential as a versatile and effective tool for tackling bias in deep learning models through synthetic data generation.

Abstract: Deep learning model effectiveness in classification tasks is often challenged by the quality and quantity of training data whenever they are affected by strong spurious correlations between specific attributes and target labels. This results in a form of bias affecting training data, which typically leads to unrecoverable weak generalization in prediction. This paper aims at facing this problem by leveraging bias amplification with generated synthetic data: we introduce Diffusing DeBias (DDB), a novel approach acting as a plug-in for common methods of unsupervised model debiasing exploiting the inherent bias-learning tendency of diffusion models in data generation. Specifically, our approach adopts conditional diffusion models to generate synthetic bias-aligned images, which replace the original training set for learning an effective bias amplifier model that we subsequently incorporate into an end-to-end and a two-step unsupervised debiasing approach. By tackling the fundamental issue of bias-conflicting training samples memorization in learning auxiliary models, typical of this type of techniques, our proposed method beats current state-of-the-art in multiple benchmark datasets, demonstrating its potential as a versatile and effective tool for tackling bias in deep learning models. Code is available at https://github.com/Malga-Vision/DiffusingDeBias

Method: Plackett-Luce Distillation (PLD) interprets teacher logits as worth scores and uses a weighted list-wise ranking loss to directly optimize a teacher-optimal class ranking where true label is first followed by classes in descending teacher confidence.

Result: PLD achieves consistent performance gains across CIFAR-100, ImageNet-1K, and MS-COCO datasets with diverse architectures and distillation objectives, including divergence-based, correlation-based, and feature-based methods.

Conclusion: PLD provides a unified framework for knowledge distillation that subsumes weighted cross-entropy and works effectively across various settings without requiring careful weight tuning.

Abstract: Knowledge distillation is a model compression technique in which a compact “student” network is trained to replicate the predictive behavior of a larger “teacher” network. In logit-based knowledge distillation, it has become the de facto approach to augment cross-entropy with a distillation term. Typically, this term is either a KL divergence that matches marginal probabilities or a correlation-based loss that captures intra- and inter-class relationships. In every case, it acts as an additional term to cross-entropy. This term has its own weight, which must be carefully tuned. In this paper, we adopt a choice-theoretic perspective and recast knowledge distillation under the Plackett-Luce model by interpreting teacher logits as “worth” scores. We introduce “Plackett-Luce Distillation (PLD)”, a weighted list-wise ranking loss. In PLD, the teacher model transfers knowledge of its full ranking of classes, weighting each ranked choice by its own confidence. PLD directly optimizes a single “teacher-optimal” ranking. The true label is placed first, followed by the remaining classes in descending teacher confidence. This process yields a convex and translation-invariant surrogate that subsumes weighted cross-entropy. Empirically, across CIFAR-100, ImageNet-1K, and MS-COCO, PLD achieves consistent gains across diverse architectures and distillation objectives, including divergence-based, correlation-based, and feature-based methods, in both homogeneous and heterogeneous teacher-student pairs.

Method: Construct ‘pointwise-undetectable’ attacks by repurposing entropy in benign model outputs to covertly transmit dangerous knowledge, using only unsuspicious benign samples collected from the model before fine-tuning.

Result: Attacks successfully elicited answers to harmful multiple-choice questions on the OpenAI fine-tuning API and evaded an enhanced monitoring system designed to detect other fine-tuning attacks.

Conclusion: Pointwise fine-tuning API defenses have fundamental limitations, and the community should develop defenses that address these uncovered weaknesses.

Abstract: LLM developers have imposed technical interventions to prevent fine-tuning misuse attacks, attacks where adversaries evade safeguards by fine-tuning the model using a public API. Previous work has established several successful attacks against specific fine-tuning API defences. In this work, we show that defences of fine-tuning APIs that seek to detect individual harmful training or inference samples (‘pointwise’ detection) are fundamentally limited in their ability to prevent fine-tuning attacks. We construct ‘pointwise-undetectable’ attacks that repurpose entropy in benign model outputs (e.g. semantic or syntactic variations) to covertly transmit dangerous knowledge. Our attacks are composed solely of unsuspicious benign samples that can be collected from the model before fine-tuning, meaning training and inference samples are all individually benign and low-perplexity. We test our attacks against the OpenAI fine-tuning API, finding they succeed in eliciting answers to harmful multiple-choice questions, and that they evade an enhanced monitoring system we design that successfully detects other fine-tuning attacks. We encourage the community to develop defences that tackle the fundamental limitations we uncover in pointwise fine-tuning API defences.

Method: Developed an analytical linear model that captures the essence of LAM learning while remaining tractable, enabling theoretical analysis of connections to PCA, data policy requirements, and strategies like data augmentation, cleaning, and auxiliary action-prediction.

Result: The analysis revealed connections between LAM and PCA, identified desiderata for data-generating policies, and justified strategies to encourage learning of controllable changes. Numerical simulations provided insights into how observation, action, and noise structures influence LAM learning.

Conclusion: The analytical framework provides theoretical foundations for understanding when and how LAMs capture action-relevant changes versus noise, offering practical guidance through data augmentation, cleaning, and auxiliary prediction methods to improve learning of controllable changes.

Abstract: Latent action models (LAMs) aim to learn action-relevant changes from unlabeled videos by compressing changes between frames as latents. However, differences between video frames can be caused by controllable changes as well as exogenous noise, leading to an important concern – do latents capture the changes caused by actions or irrelevant noise? This paper studies this issue analytically, presenting a linear model that encapsulates the essence of LAM learning, while being tractable.This provides several insights, including connections between LAM and principal component analysis (PCA), desiderata of the data-generating policy, and justification of strategies to encourage learning controllable changes using data augmentation, data cleaning, and auxiliary action-prediction. We also provide illustrative results based on numerical simulation, shedding light on the specific structure of observations, actions, and noise in data that influence LAM learning.

Method: The SB framework formulates time series synthesis as an entropic optimal interpolation transport problem between reference and target probability measures, resulting in a stochastic differential equation that captures temporal dynamics.

Result: The study provides comprehensive evaluation results comparing SB against state-of-the-art time series generation methods, assessing its strengths, limitations, and capacity to model complex temporal dependencies.

Conclusion: The results offer valuable insights into SB’s potential as a versatile and robust tool for time series generation, addressing the scarcity of studies in this application domain.

Abstract: We investigate the generative capabilities of the Schr"odinger Bridge (SB) approach for time series. The SB framework formulates time series synthesis as an entropic optimal interpolation transport problem between a reference probability measure on path space and a target joint distribution. This results in a stochastic differential equation over a finite horizon that accurately captures the temporal dynamics of the target time series. While the SB approach has been largely explored in fields like image generation, there is a scarcity of studies for its application to time series. In this work, we bridge this gap by conducting a comprehensive evaluation of the SB method’s robustness and generative performance. We benchmark it against state-of-the-art (SOTA) time series generation methods across diverse datasets, assessing its strengths, limitations, and capacity to model complex temporal dependencies. Our results offer valuable insights into the SB framework’s potential as a versatile and robust tool for time series generation.

Method: Introduce System embedded Diffusion Bridge Models (SDBs) that explicitly embed known linear measurement systems into the coefficients of matrix-valued stochastic differential equations.

Result: SDBs show consistent improvements across diverse linear inverse problems and demonstrate robust generalization under system misspecification between training and deployment.

Conclusion: SDBs offer a promising solution for real-world inverse problem applications by principled integration of measurement system information into supervised bridge methods.

Abstract: Solving inverse problems – recovering signals from incomplete or noisy measurements – is fundamental in science and engineering. Score-based generative models (SGMs) have recently emerged as a powerful framework for this task. Two main paradigms have formed: unsupervised approaches that adapt pretrained generative models to inverse problems, and supervised bridge methods that train stochastic processes conditioned on paired clean and corrupted data. While the former typically assume knowledge of the measurement model, the latter have largely overlooked this structural information. We introduce System embedded Diffusion Bridge Models (SDBs), a new class of supervised bridge methods that explicitly embed the known linear measurement system into the coefficients of a matrix-valued SDE. This principled integration yields consistent improvements across diverse linear inverse problems and demonstrates robust generalization under system misspecification between training and deployment, offering a promising solution to real-world applications.

Method: The authors parameterize dense linear RNNs as fixed-points of parallelizable diagonal linear RNNs, allowing models to naturally trade expressivity for efficiency at a fixed parameter count.

Result: The proposed models achieve state-of-the-art results on state-tracking benchmarks A5 and S5, while matching performance on copying and other tasks.

Conclusion: This approach successfully bridges the gap between expressivity and efficiency in linear RNNs, providing a flexible framework that can adapt to different computational requirements while maintaining strong performance across various tasks.

Abstract: Linear recurrent neural networks (RNNs) and state-space models (SSMs) such as Mamba have become promising alternatives to softmax-attention as sequence mixing layers in Transformer architectures. Current models, however, do not exhibit the full state-tracking expressivity of RNNs because they rely on channel-wise (i.e. diagonal) sequence mixing. In this paper, we investigate parameterizations of a large class of dense linear RNNs as fixed-points of parallelizable diagonal linear RNNs. The resulting models can naturally trade expressivity for efficiency at a fixed number of parameters and achieve state-of-the-art results on the state-tracking benchmarks $A_5$ and $S_5$, while matching performance on copying and other tasks.

Method: Applies action chunking to TD-based RL methods by running RL in a ‘chunked’ action space, predicting sequences of future actions rather than single actions, and using unbiased n-step backups.

Result: Q-chunking demonstrates strong offline performance and online sample efficiency, outperforming prior best offline-to-online methods on various long-horizon sparse-reward manipulation tasks.

Conclusion: Action chunking can be effectively applied to TD-based RL methods to improve exploration and learning efficiency in offline-to-online settings for challenging long-horizon tasks.

Abstract: We present Q-chunking, a simple yet effective recipe for improving reinforcement learning (RL) algorithms for long-horizon, sparse-reward tasks. Our recipe is designed for the offline-to-online RL setting, where the goal is to leverage an offline prior dataset to maximize the sample-efficiency of online learning. Effective exploration and sample-efficient learning remain central challenges in this setting, as it is not obvious how the offline data should be utilized to acquire a good exploratory policy. Our key insight is that action chunking, a technique popularized in imitation learning where sequences of future actions are predicted rather than a single action at each timestep, can be applied to temporal difference (TD)-based RL methods to mitigate the exploration challenge. Q-chunking adopts action chunking by directly running RL in a ‘chunked’ action space, enabling the agent to (1) leverage temporally consistent behaviors from offline data for more effective online exploration and (2) use unbiased $n$-step backups for more stable and efficient TD learning. Our experimental results demonstrate that Q-chunking exhibits strong offline performance and online sample efficiency, outperforming prior best offline-to-online methods on a range of long-horizon, sparse-reward manipulation tasks.

Method: Developed COSIMO architecture derived from partial differential equations on simplicial complexes, with theoretical and experimental analysis of stability under simplicial perturbations.

Result: COSIMO achieves competitive performance compared to state-of-the-art SNNs in complex and noisy environments, with better control over over-smoothing phenomenon.

Conclusion: COSIMO provides a continuous approach to simplicial neural networks that offers improved stability and over-smoothing control while maintaining competitive performance.

Abstract: Simplicial complexes provide a powerful framework for modeling higher-order interactions in structured data, making them particularly suitable for applications such as trajectory prediction and mesh processing. However, existing simplicial neural networks (SNNs), whether convolutional or attention-based, rely primarily on discrete filtering techniques, which can be restrictive. In contrast, partial differential equations (PDEs) on simplicial complexes offer a principled approach to capture continuous dynamics in such structures. In this work, we introduce continuous simplicial neural network (COSIMO), a novel SNN architecture derived from PDEs on simplicial complexes. We provide theoretical and experimental justifications of COSIMO’s stability under simplicial perturbations. Furthermore, we investigate the over-smoothing phenomenon, a common issue in geometric deep learning, demonstrating that COSIMO offers better control over this effect than discrete SNNs. Our experiments on real-world datasets demonstrate that COSIMO achieves competitive performance compared to state-of-the-art SNNs in complex and noisy environments. The implementation codes are available in https://github.com/ArefEinizade2/COSIMO.

Method: Extends causal estimation under hidden confounding by leveraging proxy variables and do-calculus, training once per dataset on observational data with causal graphs.

Result: Outperforms existing approaches on diverse settings including the Ecoli70 dataset with 3 hidden confounders, tens of observed variables, and hundreds of causal queries.

Conclusion: DeCaFlow provides correct estimates for all identifiable causal queries and counterfactuals, demonstrating broad applicability to any causal graph with hidden confounders.

Abstract: We introduce DeCaFlow, a deconfounding causal generative model. Training once per dataset using just observational data and the underlying causal graph, DeCaFlow enables accurate causal inference on continuous variables under the presence of hidden confounders. Specifically, we extend previous results on causal estimation under hidden confounding to show that a single instance of DeCaFlow provides correct estimates for all causal queries identifiable with do-calculus, leveraging proxy variables to adjust for the causal effects when do-calculus alone is insufficient. Moreover, we show that counterfactual queries are identifiable as long as their interventional counterparts are identifiable, and thus are also correctly estimated by DeCaFlow. Our empirical results on diverse settings (including the Ecoli70 dataset, with 3 independent hidden confounders, tens of observed variables and hundreds of causal queries) show that DeCaFlow outperforms existing approaches, while demonstrating its out-of-the-box applicability to any given causal graph. An implementation can be found in https://github.com/aalmodovares/DeCaFlow

Method: Examined various open-sourced LLMs, analyzed calibration degradation after instruction tuning, applied label smoothing as regularization, and designed a custom kernel for memory-efficient loss computation.

Result: Found significant calibration degradation after instruction tuning across all tested LLMs. Label smoothing effectively maintains calibration but is less effective for large vocabulary LLMs due to overconfidence issues related to hidden and vocabulary sizes.

Conclusion: Label smoothing is a practical solution for maintaining LLM calibration during SFT, though its effectiveness diminishes with large vocabulary models. The proposed custom kernel enables efficient memory usage for smoothed loss computation.

Abstract: Recent advances in natural language processing (NLP) have opened up greater opportunities to enable fine-tuned large language models (LLMs) to behave as more powerful interactive agents through improved instruction-following ability. However, understanding how this impacts confidence calibration for reliable model output has not been researched in full. In this work, we examine various open-sourced LLMs, identifying significant calibration degradation after instruction tuning in each. Seeking a practical solution, we look towards label smoothing, which has been shown as an effective method to regularize for overconfident predictions but has yet to be widely adopted in the supervised fine-tuning (SFT) of LLMs. We first provide insight as to why label smoothing is sufficient to maintain calibration throughout the SFT process. However, settings remain where the effectiveness of smoothing is severely diminished, in particular the case of large vocabulary LLMs (LV-LLMs). We posit the cause to stem from the ability to become over-confident, which has a direct relationship with the hidden size and vocabulary size, and justify this theoretically and experimentally. Finally, we address an outstanding issue regarding the memory footprint of the cross-entropy loss computation in the label smoothed loss setting, designing a customized kernel to dramatically reduce memory consumption without sacrificing speed or performance in comparison to existing solutions for non-smoothed losses.

Method: Lower bound uses topological argument based on local Borsuk-Ulam theorem; upper bound constructs list-replicable learning rule using SVM generalization properties.

Result: Proved list replicability number of d-dimensional γ-margin half-spaces satisfies d/2+1 ≤ LR(H^d_γ) ≤ d, resolving 5 open problems including unbounded Littlestone dimension for disambiguations, unbounded communication complexity, separation between randomized and pseudo-deterministic communication, and maximum list-replicability number.

Conclusion: List replicability grows with dimension for margin half-spaces, establishing fundamental limitations in learning theory and communication complexity, with implications for disambiguation problems and complexity separations.

Abstract: We prove that the list replicability number of $d$-dimensional $\gamma$-margin half-spaces satisfies [ \frac{d}{2}+1 \le \mathrm{LR}(H^d_\gamma) \le d, ] which grows with dimension. This resolves several open problems: $\bullet$ Every disambiguation of infinite-dimensional large-margin half-spaces to a total concept class has unbounded Littlestone dimension, answering an open question of Alon, Hanneke, Holzman, and Moran (FOCS ‘21). $\bullet$ Every disambiguation of the Gap Hamming Distance problem in the large gap regime has unbounded public-coin randomized communication complexity. This answers an open question of Fang, G"o"os, Harms, and Hatami (STOC ‘25). $\bullet$ There is a separation of $O(1)$ vs $\omega(1)$ between randomized and pseudo-deterministic communication complexity. $\bullet$ The maximum list-replicability number of any finite set of points and homogeneous half-spaces in $d$-dimensional Euclidean space is $d$, resolving a problem of Chase, Moran, and Yehudayoff (FOCS ‘23). $\bullet$ There exists a partial concept class with Littlestone dimension $1$ such that all its disambiguations have infinite Littlestone dimension. This resolves a problem of Cheung, H. Hatami, P. Hatami, and Hosseini (ICALP ‘23). Our lower bound follows from a topological argument based on a local Borsuk-Ulam theorem. For the upper bound, we construct a list-replicable learning rule using the generalization properties of SVMs.

Method: Developed SICKLE framework with maximum entropy (MaxEnt) sampling approach, scalable training, and energy benchmarking. Compared MaxEnt with random and phase-space sampling on turbulence DNS datasets.

Result: Subsampling as preprocessing step improved model accuracy in many cases and substantially lowered energy consumption with up to 38x reductions.

Conclusion: Intelligent subsampling can enable better model training with significantly less data while reducing energy consumption.

Abstract: With the end of Moore’s law and Dennard scaling, efficient training increasingly requires rethinking data volume. Can we train better models with significantly less data via intelligent subsampling? To explore this, we develop SICKLE, a sparse intelligent curation framework for efficient learning, featuring a novel maximum entropy (MaxEnt) sampling approach, scalable training, and energy benchmarking. We compare MaxEnt with random and phase-space sampling on large direct numerical simulation (DNS) datasets of turbulence. Evaluating SICKLE at scale on Frontier, we show that subsampling as a preprocessing step can, in many cases, improve model accuracy and substantially lower energy consumption, with observed reductions of up to 38x.

Method: Proposed a relative value iteration algorithm for planning and two model-free Q-learning algorithms: one based on multi-level Monte Carlo method and another off-policy algorithm for utility-based shortfall risk measures.

Result: Both the RVI and MLMC-based Q-learning algorithms are proven to converge to optimality. Numerical experiments validate the analysis and demonstrate the off-policy algorithm’s convergence.

Conclusion: The approach successfully enables identification of policies that are finely tuned to the agent’s specific risk-awareness, extending traditional risk-neutral methods to accommodate dynamic risk measures.

Abstract: For continuing tasks, average cost Markov decision processes have well-documented value and can be solved using efficient algorithms. However, it explicitly assumes that the agent is risk-neutral. In this work, we extend risk-neutral algorithms to accommodate the more general class of dynamic risk measures. Specifically, we propose a relative value iteration (RVI) algorithm for planning and design two model-free Q-learning algorithms, namely a generic algorithm based on the multi-level Monte Carlo (MLMC) method, and an off-policy algorithm dedicated to utility-based shortfall risk measures. Both the RVI and MLMC-based Q-learning algorithms are proven to converge to optimality. Numerical experiments validate our analysis, confirm empirically the convergence of the off-policy algorithm, and demonstrate that our approach enables the identification of policies that are finely tuned to the intricate risk-awareness of the agent that they serve.

Method: Leverages the insight that layer sensitivity changes dynamically across decoding steps, and introduces DP-LLM which dynamically assigns precision to each layer based on input values.

Result: Experimental results across multiple models and benchmarks show DP-LLM achieves superior performance-latency trade-off, outperforming prior approaches.

Conclusion: Dynamic precision assignment based on input values and changing layer sensitivity across decoding steps provides an effective solution for on-device LLM adaptation with varying runtime constraints.

Abstract: How can we effectively handle queries for on-device large language models (LLMs) with varying runtime constraints, such as latency and accuracy? Multi-scale quantization addresses this challenge by enabling memory-efficient runtime model adaptation of LLMs through the overlaying of multiple model variants quantized to different bitwidths. Meanwhile, an important question still remains open-ended: how can models be properly configured to match a target precision or latency? While mixed-precision offers a promising solution, we take this further by leveraging the key observation that the sensitivity of each layer dynamically changes across decoding steps. Building on this insight, we introduce DP-LLM, a novel mechanism that dynamically assigns precision to each layer based on input values. Experimental results across multiple models and benchmarks demonstrate that DP-LLM achieves a superior performance-latency trade-off, outperforming prior approaches.

Method: Formulate three TeamFormation variants with different cost functions using QUBO, and evaluate two general-purpose solution methods including graph neural networks for transfer learning.

Result: QUBO-based solutions are at least as good as established baselines, and GNN-based approaches enable effective transfer learning between problem instances.

Conclusion: The unified QUBO formulation successfully captures various team formation cost definitions and enables both competitive performance and transfer learning capabilities.

Abstract: The team formation problem assumes a set of experts and a task, where each expert has a set of skills and the task requires some skills. The objective is to find a set of experts that maximizes coverage of the required skills while simultaneously minimizing the costs associated with the experts. Different definitions of cost have traditionally led to distinct problem formulations and algorithmic solutions. We introduce the unified TeamFormation formulation that captures all cost definitions for team formation problems that balance task coverage and expert cost. Specifically, we formulate three TeamFormation variants with different cost functions using quadratic unconstrained binary optimization (QUBO), and we evaluate two distinct general-purpose solution methods. We show that solutions based on the QUBO formulations of TeamFormation problems are at least as good as those produced by established baselines. Furthermore, we show that QUBO-based solutions leveraging graph neural networks can effectively learn representations of experts and skills to enable transfer learning, allowing node embeddings from one problem instance to be efficiently applied to another.

Method: Introduces a reversed-sign reward term in the loss function with power-law decay schedule, formalizes spectral safety conditions and trust-region constraints, and implements lightweight safeguards combining projection operators with gradient clipping.

Result: Empirical results show anti-regularization mitigates underfitting in regression and classification while preserving generalization and improving calibration. Ablation studies confirm decay schedule and safeguards are essential.

Conclusion: Anti-regularization provides a simple, reproducible procedure that integrates into standard empirical risk minimization, enabling robust learning under limited data by intervening only when necessary and vanishing otherwise.

Abstract: Anti-regularization introduces a reward term with a reversed sign into the loss function, deliberately amplifying model expressivity in small-sample regimes while ensuring that the intervention gradually vanishes as the sample size grows through a power-law decay schedule. We formalize spectral safety conditions and trust-region constraints, and we design a lightweight safeguard that combines a projection operator with gradient clipping to guarantee stable intervention. Theoretical analysis extends to linear smoothers and the Neural Tangent Kernel regime, providing practical guidance on the choice of decay exponents through the balance between empirical risk and variance. Empirical results show that Anti-regularization mitigates underfitting in both regression and classification while preserving generalization and improving calibration. Ablation studies confirm that the decay schedule and safeguards are essential to avoiding overfitting and instability. As an alternative, we also propose a degrees-of-freedom targeting schedule that maintains constant per-sample complexity. Anti-regularization constitutes a simple and reproducible procedure that integrates seamlessly into standard empirical risk minimization pipelines, enabling robust learning under limited data and resource constraints by intervening only when necessary and vanishing otherwise.

Method: Leverages standard client model updates as pseudo-gradients and applies their negation (appropriately scaled) to the global model when a client needs to be forgotten, supporting concurrent unlearning requests.

Result: Achieves state-of-the-art forgetting effectiveness and recovery time on CIFAR-10, CIFAR-100, and ProstateMRI datasets using various neural architectures, without additional computational or communication overhead.

Conclusion: PUF provides a practical and efficient federated unlearning solution that seamlessly integrates with existing FL workflows while maintaining performance.

Abstract: The right to be forgotten is a fundamental principle of privacy-preserving regulations and extends to Machine Learning (ML) paradigms such as Federated Learning (FL). While FL enhances privacy by enabling collaborative model training without sharing private data, trained models still retain the influence of training data. Federated Unlearning (FU) methods recently proposed often rely on impractical assumptions for real-world FL deployments, such as storing client update histories or requiring access to a publicly available dataset. To address these constraints, this paper introduces a novel method that leverages negated Pseudo-gradients Updates for Federated Unlearning (PUF). Our approach only uses standard client model updates, which are employed during regular FL rounds, and interprets them as pseudo-gradients. When a client needs to be forgotten, we apply the negation of their pseudo-gradients, appropriately scaled, to the global model. Unlike state-of-the-art mechanisms, PUF seamlessly integrates with FL workflows, incurs no additional computational and communication overhead beyond standard FL rounds, and supports concurrent unlearning requests. We extensively evaluated the proposed method on two well-known benchmark image classification datasets (CIFAR-10 and CIFAR-100) and a real-world medical imaging dataset for segmentation (ProstateMRI), using three different neural architectures: two residual networks and a vision transformer. The experimental results across various settings demonstrate that PUF achieves state-of-the-art forgetting effectiveness and recovery time, without relying on any additional assumptions.

Method: Used stochastic differential equation analysis of optimization dynamics in tractable setups, derived theoretical guarantees, and conducted extensive experiments on CelebA, FairFace, and MS-COCO datasets across various tasks using multiple fairness definitions.

Result: Adaptive optimizers (RMSProp, Adam) consistently achieve better group fairness than SGD across multiple datasets and fairness metrics, with RMSProp showing higher likelihood of converging to fairer minima and providing fairer parameter updates.

Conclusion: Adaptive updates serve as a crucial yet overlooked mechanism for promoting fair outcomes in deep learning, highlighting the importance of optimizer selection for fairness considerations.

Abstract: We study whether and how the choice of optimization algorithm can impact group fairness in deep neural networks. Through stochastic differential equation analysis of optimization dynamics in an analytically tractable setup, we demonstrate that the choice of optimization algorithm indeed influences fairness outcomes, particularly under severe imbalance. Furthermore, we show that when comparing two categories of optimizers, adaptive methods and stochastic methods, RMSProp (from the adaptive category) has a higher likelihood of converging to fairer minima than SGD (from the stochastic category). Building on this insight, we derive two new theoretical guarantees showing that, under appropriate conditions, RMSProp exhibits fairer parameter updates and improved fairness in a single optimization step compared to SGD. We then validate these findings through extensive experiments on three publicly available datasets, namely CelebA, FairFace, and MS-COCO, across different tasks as facial expression recognition, gender classification, and multi-label classification, using various backbones. Considering multiple fairness definitions including equalized odds, equal opportunity, and demographic parity, adaptive optimizers like RMSProp and Adam consistently outperform SGD in terms of group fairness, while maintaining comparable predictive accuracy. Our results highlight the role of adaptive updates as a crucial yet overlooked mechanism for promoting fair outcomes. We release the source code at: https://github.com/Mkolahdoozi/Some-Optimizers-Are-More-Equal.

Method: Develop transformer-based foundation models specifically designed for ECG data processing.

Result: Achieved impressive performance across multiple downstream ECG applications.

Conclusion: Transformer models are effective for ECG analysis and show promise for various clinical applications.

Abstract: Transformer-based foundation models for Electrocardiograms (ECGs) have recently achieved impressive performance in many downstream applications.

Method: Extends PROM-ANN architecture with discrete physics-informed residual loss using FEM residuals, parameter-agnostic loss for non-linear problems, and architectural modifications for fast-decaying singular values.

Result: Outperforms POD by orders of magnitude in snapshot reconstruction accuracy, maintains reasonable training times for non-linear problems, and modestly narrows gap between data reconstruction and ROM accuracy.

Conclusion: FEM residuals play critical role in ROM construction, showing untapped potential for residual-driven optimization and calling for exploration of architectures beyond PROM-ANN.

Abstract: This paper presents a physics-informed training framework for projection-based Reduced Order Models (ROMs). We extend the PROM-ANN architecture by complementing snapshot-based training with a FEM-based, discrete physics-informed residual loss, bridging the gap between traditional projection-based ROMs and physics-informed neural networks (PINNs). Unlike conventional PINNs that rely on analytical PDEs, our approach leverages FEM residuals to guide the learning of the ROM approximation manifold. Key contributions include: (1) a parameter-agnostic, discrete residual loss applicable to non-linear problems, (2) an architectural modification to PROM-ANN improving accuracy for fast-decaying singular values, and (3) an empirical study on the proposed physics informed training process for ROMs. The method is demonstrated on a non-linear hyperelasticity problem, simulating a rubber cantilever under multi-axial loads. The main accomplishment in regards to the proposed residual-based loss is its applicability on non-linear problems by interfacing with FEM software while maintaining reasonable training times. The modified PROM-ANN outperforms POD by orders of magnitude in snapshot reconstruction accuracy, while the original formulation is not able to learn a proper mapping for this use-case. Finally, the application of physics informed training in ANN-PROM modestly narrows the gap between data reconstruction and ROM accuracy, however it highlights the untapped potential of the proposed residual-driven optimization for future ROM development. This work underscores the critical role of FEM residuals in ROM construction and calls for further exploration on architectures beyond PROM-ANN.

Method: Applied LPCMCI framework with Gaussian Process Distance Correlation (GPDC) for causal discovery on quarterly US data (1970-2021), then used Chronos LLM framework for zero-shot probabilistic forecasting of unemployment.

Result: Found unidirectional causal link from economic growth to GDP, limited connectivity of inflation, strong autoregressive dependence in unemployment. Achieved accurate 1-2 quarter ahead unemployment forecasts with 90% confidence intervals.

Conclusion: Combining causal structure learning with probabilistic language models provides valuable insights for economic policy and enhances forecasting robustness through uncertainty-aware predictions.

Abstract: This paper presents a methodological approach to financial time series analysis by combining causal discovery and uncertainty-aware forecasting. As a case study, we focus on four key U.S. macroeconomic indicators – GDP, economic growth, inflation, and unemployment – and we apply the LPCMCI framework with Gaussian Process Distance Correlation (GPDC) to uncover dynamic causal relationships in quarterly data from 1970 to 2021. Our results reveal a robust unidirectional causal link from economic growth to GDP and highlight the limited connectivity of inflation, suggesting the influence of latent factors. Unemployment exhibits strong autoregressive dependence, motivating its use as a case study for probabilistic forecasting. Leveraging the Chronos framework, a large language model trained for time series, we perform zero-shot predictions on unemployment. This approach delivers accurate forecasts one and two quarters ahead, without requiring task-specific training. Crucially, the model’s uncertainty-aware predictions yield 90% confidence intervals, enabling effective anomaly detection through statistically principled deviation analysis. This study demonstrates the value of combining causal structure learning with probabilistic language models to inform economic policy and enhance forecasting robustness.

Method: Combines a large language model and probabilistic programming language to define joint distributions over formal models, latent variables, and data. Uses an inference recipe that generates many formal models from the LLM, performs approximate inference on each, and does a weighted average using self-normalized importance sampling, MCMC, and importance-weighted variational inference.

Result: The method produces sensible predictions using only data and an informal problem description, without requiring specification of a formal model.

Conclusion: This approach enables domain experts to use Bayesian methods without needing to write formal models, making probabilistic programming more accessible.

Abstract: Many domain experts do not have the time or expertise to write formal Bayesian models. This paper takes an informal problem description as input, and combines a large language model and a probabilistic programming language to define a joint distribution over formal models, latent variables, and data. A posterior over latent variables follows by conditioning on observed data and integrating over formal models. This presents a challenging inference problem. We suggest an inference recipe that amounts to generating many formal models from the large language model, performing approximate inference on each, and then doing a weighted average. This is justified and analyzed as a combination of self-normalized importance sampling, MCMC, and importance-weighted variational inference. Experimentally, this produces sensible predictions from only data and an informal problem description, without the need to specify a formal model.

Method: Formulates optimization program for optimal steering using KL-divergence to find nearest ideal distribution, with efficient algorithms for parametric families. Applies affine steering to LLM representations and internal representations.

Result: Empirical results show improved fairness without diminishing utility on synthetic and real-world datasets. Successfully reduces bias in multi-class classification tasks like occupation prediction from biographies.

Conclusion: Optimal steering techniques effectively address bias in ML systems by transforming distributions to ideal forms that guarantee fair outcomes while maintaining or even improving utility.

Abstract: To fix the ‘bias in, bias out’ problem in fair machine learning, it is important to steer feature distributions of data or internal representations of Large Language Models (LLMs) to ideal ones that guarantee group-fair outcomes. Previous work on fair generative models and representation steering could greatly benefit from provable fairness guarantees on the model output. We define a distribution as ideal if the minimizer of any cost-sensitive risk on it is guaranteed to have exact group-fair outcomes (e.g., demographic parity, equal opportunity)-in other words, it has no fairness-utility trade-off. We formulate an optimization program for optimal steering by finding the nearest ideal distribution in KL-divergence, and provide efficient algorithms for it when the underlying distributions come from well-known parametric families (e.g., normal, log-normal). Empirically, our optimal steering techniques on both synthetic and real-world datasets improve fairness without diminishing utility (and sometimes even improve utility). We demonstrate affine steering of LLM representations to reduce bias in multi-class classification, e.g., occupation prediction from a short biography in Bios dataset (De-Arteaga et al.). Furthermore, we steer internal representations of LLMs towards desired outputs so that it works equally well across different groups.

Method: The study uses theoretically supported adaptive MMD measures within the Ditto pFL framework to address data heterogeneity, particularly focusing on feature heterogeneity.

Result: The adaptive MMD measures significantly improve model performance across various tasks, especially those with pronounced feature heterogeneity, and show similar improvements when applied to other pFL techniques on multiple datasets.

Conclusion: The results motivate using constraints specifically tailored to different types of heterogeneity expected in federated learning systems.

Abstract: Federated learning (FL) is an effective and widely used approach to training deep learning models on decentralized datasets held by distinct clients. FL also strengthens both security and privacy protections for training data. Common challenges associated with statistical heterogeneity between distributed datasets have spurred significant interest in personalized FL (pFL) methods, where models combine aspects of global learning with local modeling specific to each client’s unique characteristics. This work investigates the efficacy of theoretically supported, adaptive MMD measures in pFL, primarily focusing on the Ditto framework, a state-of-the-art technique for distributed data heterogeneity. The use of such measures significantly improves model performance across a variety of tasks, especially those with pronounced feature heterogeneity. Additional experiments demonstrate that such measures are directly applicable to other pFL techniques and yield similar improvements across a number of datasets. Finally, the results motivate the use of constraints tailored to the various kinds of heterogeneity expected in FL systems.

Method: Multi-Level Cross-Modal Knowledge Distillation with Dual-Injected Knowledge Distillation - a bidirectional mechanism that guides thermal student with RGB features and validates learned representations through closed-loop supervision.

Result: Sets new state-of-the-art on public thermal FLD benchmarks, significantly outperforming previous methods while drastically reducing computational overhead.

Conclusion: The proposed MLCM-KD framework with DIKD enables robust and efficient knowledge transfer between RGB and thermal modalities, creating accurate and practical thermal FLD models.

Abstract: Facial Landmark Detection (FLD) in thermal imagery is critical for applications in challenging lighting conditions, but it is hampered by the lack of rich visual cues. Conventional cross-modal solutions, like feature fusion or image translation from RGB data, are often computationally expensive or introduce structural artifacts, limiting their practical deployment. To address this, we propose Multi-Level Cross-Modal Knowledge Distillation (MLCM-KD), a novel framework that decouples high-fidelity RGB-to-thermal knowledge transfer from model compression to create both accurate and efficient thermal FLD models. A central challenge during knowledge transfer is the profound modality gap between RGB and thermal data, where traditional unidirectional distillation fails to enforce semantic consistency across disparate feature spaces. To overcome this, we introduce Dual-Injected Knowledge Distillation (DIKD), a bidirectional mechanism designed specifically for this task. DIKD establishes a connection between modalities: it not only guides the thermal student with rich RGB features but also validates the student’s learned representations by feeding them back into the frozen teacher’s prediction head. This closed-loop supervision forces the student to learn modality-invariant features that are semantically aligned with the teacher, ensuring a robust and profound knowledge transfer. Experiments show that our approach sets a new state-of-the-art on public thermal FLD benchmarks, notably outperforming previous methods while drastically reducing computational overhead.

Method: Theoretical analysis using o-minimal structures geometry, proving convergence/divergence properties for gradient flows, with numerical experiments to validate findings.

Result: Proved gradient flows either converge to critical points or diverge to infinity while loss converges to asymptotic critical values. For polynomial targets with good initialization, gradient flows diverge to infinity.

Conclusion: Gradient flows in neural networks exhibit either convergence to critical points or divergence to infinity, with the latter occurring for polynomial targets with proper initialization, as confirmed by both theory and experiments.

Abstract: We study gradient flows for loss landscapes of fully connected feedforward neural networks with commonly used continuously differentiable activation functions such as the logistic, hyperbolic tangent, softplus or GELU function. We prove that the gradient flow either converges to a critical point or diverges to infinity while the loss converges to an asymptotic critical value. Moreover, we prove the existence of a threshold $\varepsilon>0$ such that the loss value of any gradient flow initialized at most $\varepsilon$ above the optimal level converges to it. For polynomial target functions and sufficiently big architecture and data set, we prove that the optimal loss value is zero and can only be realized asymptotically. From this setting, we deduce our main result that any gradient flow with sufficiently good initialization diverges to infinity. Our proof heavily relies on the geometry of o-minimal structures. We confirm these theoretical findings with numerical experiments and extend our investigation to more realistic scenarios, where we observe an analogous behavior.

Method: Uses Difficulty-ADAptive GRPO (ADAGRPO) algorithm with difficulty-adaptive positive replay, failure curriculum filtering, and shortest-path reward adjustment for multi-turn tasks.

Result: Applied to Qwen2.5-VL-7B-Instruct and GLM-4.1V-9B-Base models, achieving state-of-the-art success rates of 80.2% on AndroidWorld and 53.6% on AndroidLab.

Conclusion: MobileRL framework stabilizes RL training, improves sample efficiency, and generates strong performance across diverse mobile apps and tasks.

Abstract: Building general-purpose graphical user interface (GUI) agents has become increasingly promising with the progress in vision language models. However, developing effective mobile GUI agents with reinforcement learning (RL) remains challenging due to the heavy-tailed distribution of task difficulty and the inefficiency of large-scale environment sampling. We present an online agentic reinforcement learning framework MobileRL to enhance GUI agents in mobile environments. Its core component is the Difficulty-ADAptive GRPO (ADAGRPO) algorithm. In ADAGRPO, we design difficulty-adaptive positive replay and failure curriculum filtering to adapt the model to different task difficulties. We introduce the shortest-path reward adjustment strategy to reshape rewards concerning the task length in multi-turn agentic tasks. Those strategies jointly stabilize RL training, improve sample efficiency, and generate strong performance across diverse mobile apps and tasks. We apply MOBILERL to two open models (Qwen2.5-VL-7B-Instruct and GLM-4.1V-9B-Base). The resultant MOBILERL-9B model achieves state-of-the-art results in terms of success rates on both AndroidWorld (80.2%) and AndroidLab (53.6%). The MOBILERL framework is open-sourced at: https://github.com/THUDM/MobileRL.

Method: ZEUS uses pre-training on synthetic datasets from a latent-variable prior, decomposing complex datasets into meaningful components for clustering without additional training or fine-tuning.

Result: Experimental results show ZEUS performs competitively with traditional clustering algorithms and recent deep learning methods, while being significantly faster and more user-friendly.

Conclusion: ZEUS is the first zero-shot method for unsupervised embedding generation in tabular data, offering effective clustering without per-dataset tuning or user intervention.

Abstract: Clustering tabular data remains a significant open challenge in data analysis and machine learning. Unlike for image data, similarity between tabular records often varies across datasets, making the definition of clusters highly dataset-dependent. Furthermore, the absence of supervised signals complicates hyperparameter tuning in deep learning clustering methods, frequently resulting in unstable performance. To address these issues and reduce the need for per-dataset tuning, we adopt an emerging approach in deep learning: zero-shot learning. We propose ZEUS, a self-contained model capable of clustering new datasets without any additional training or fine-tuning. It operates by decomposing complex datasets into meaningful components that can then be clustered effectively. Thanks to pre-training on synthetic datasets generated from a latent-variable prior, it generalizes across various datasets without requiring user intervention. To the best of our knowledge, ZEUS is the first zero-shot method capable of generating embeddings for tabular data in a fully unsupervised manner. Experimental results demonstrate that it performs on par with or better than traditional clustering algorithms and recent deep learning-based methods, while being significantly faster and more user-friendly.

Method: Analyzes state convergence behavior linked to transition matrix A spectrum, then proposes spectrum scaling approach to selectively modulate A matrices in pre-trained Mamba models.

Result: Spectrum scaling significantly improves long-context performance where simple Δ_t modulation fails, validating the connection between state convergence and transition matrix spectrum.

Conclusion: The work provides insights into length generalization limitations of state-space models and offers a practical solution through spectrum scaling for better long-context performance.

Abstract: The quadratic complexity of the attention mechanism in Transformer models has motivated the development of alternative architectures with sub-quadratic scaling, such as state-space models. Among these, Mamba has emerged as a leading architecture, achieving state-of-the-art results across a range of language modeling tasks. However, Mamba’s performance significantly deteriorates when applied to contexts longer than those seen during pre-training, revealing a sharp sensitivity to context length extension. Through detailed analysis, we attribute this limitation to the out-of-distribution behaviour of its state-space dynamics, particularly within the parameterization of the state transition matrix $\mathbf{A}$. Unlike recent works which attribute this sensitivity to the vanished accumulation of discretization time steps, $\exp(-\sum_{t=1}^N\Delta_t)$, we establish a connection between state convergence behavior as the input length approaches infinity and the spectrum of the transition matrix $\mathbf{A}$, offering a well-founded explanation of its role in length extension. Next, to overcome this challenge, we propose an approach that applies spectrum scaling to pre-trained Mamba models to enable robust long-context generalization by selectively modulating the spectrum of $\mathbf{A}$ matrices in each layer. We show that this can significantly improve performance in settings where simply modulating $\Delta_t$ fails, validating our insights and providing avenues for better length generalization of state-space models with structured transition matrices.

Method: Replace standard Gaussian prior with learnable distribution optimized via behavior-regularized objective, apply behavior regularization in latent space, and directly generate high-value trajectories without reward optimization of diffusion model.

Result: PG outperforms state-of-the-art diffusion policies and planners across diverse long-horizon offline RL benchmarks, eliminating the need for costly multiple candidate sampling at inference.

Conclusion: Prior Guidance provides an efficient and effective framework for diffusion-based offline reinforcement learning that addresses key limitations of existing guided sampling approaches.

Abstract: Diffusion models have recently gained prominence in offline reinforcement learning due to their ability to effectively learn high-performing, generalizable policies from static datasets. Diffusion-based planners facilitate long-horizon decision-making by generating high-quality trajectories through iterative denoising, guided by return-maximizing objectives. However, existing guided sampling strategies such as Classifier Guidance, Classifier-Free Guidance, and Monte Carlo Sample Selection either produce suboptimal multi-modal actions, struggle with distributional drift, or incur prohibitive inference-time costs. To address these challenges, we propose Prior Guidance (PG), a novel guided sampling framework that replaces the standard Gaussian prior of a behavior-cloned diffusion model with a learnable distribution, optimized via a behavior-regularized objective. PG directly generates high-value trajectories without costly reward optimization of the diffusion model itself, and eliminates the need to sample multiple candidates at inference for sample selection. We present an efficient training strategy that applies behavior regularization in latent space, and empirically demonstrate that PG outperforms state-of-the-art diffusion policies and planners across diverse long-horizon offline RL benchmarks.Our code is available at https://github.com/ku-dmlab/PG.

Method: Uses data-adaptive learned orthogonal transforms that compact over 90% of signal energy into the first half of dimensions, enabling early candidate pruning. Integrates with existing ANNS methods (IVFPQ/Flat, HNSW, MRPT, Annoy) without index modification using level-major memory layouts, SIMD-vectorized partial distance computations, and cache-aware access patterns.

Result: Achieves 2-30× end-to-end speedup across diverse datasets (CIFAR-10, GIST, OpenAI’s Ada 2 and Large 3) with no recall loss.

Conclusion: PANORAMA effectively tackles the ANNS verification bottleneck through learned transforms and optimization techniques, providing significant performance improvements while maintaining accuracy.

Abstract: Approximate Nearest-Neighbor Search (ANNS) efficiently finds data items whose embeddings are close to that of a given query in a high-dimensional space, aiming to balance accuracy with speed. Used in recommendation systems, image and video retrieval, natural language processing, and retrieval-augmented generation (RAG), ANNS algorithms such as IVFPQ, HNSW graphs, Annoy, and MRPT utilize graph, tree, clustering, and quantization techniques to navigate large vector spaces. Despite this progress, ANNS systems spend up to 99% of query time to compute distances in their final refinement phase. In this paper, we present PANORAMA, a machine learning-driven approach that tackles the ANNS verification bottleneck through data-adaptive learned orthogonal transforms that facilitate the accretive refinement of distance bounds. Such transforms compact over 90% of signal energy into the first half of dimensions, enabling early candidate pruning with partial distance computations. We integrate PANORAMA into state-of-the-art ANNS methods, namely IVFPQ/Flat, HNSW, MRPT, and Annoy, without index modification, using level-major memory layouts, SIMD-vectorized partial distance computations, and cache-aware access patterns. Experiments across diverse datasets – from image-based CIFAR-10 and GIST to modern embedding spaces including OpenAI’s Ada 2 and Large 3 – demonstrate that PANORAMA affords a 2–30$\times$ end-to-end speedup with no recall loss.

Method: Based on theoretical analysis showing optimal efficiency depends on attention weight matrix rank, FlashBias uses low-rank compressed sensing theory to provide fast-exact computation for common biases and fast-accurate approximation for general biases.

Result: FlashBias achieves 1.5× speedup for Pairformer in AlphaFold 3 and over 2× speedup for attention with bias in vision and language models without accuracy loss.

Conclusion: FlashBias effectively addresses the efficiency bottleneck in attention with bias computation, enabling significant performance improvements while maintaining accuracy across various domains.

Abstract: Attention with bias, which extends standard attention by introducing prior knowledge as an additive bias matrix to the query-key scores, has been widely deployed in vision, language, protein-folding and other advanced scientific models, underscoring its status as a key evolution of this foundational module. However, introducing bias terms creates a severe efficiency bottleneck in attention computation. It disrupts the tightly fused memory-compute pipeline that underlies the speed of accelerators like FlashAttention, thereby stripping away most of their performance gains and leaving biased attention computationally expensive. Surprisingly, despite its common usage, targeted efficiency optimization for attention with bias remains absent, which seriously hinders its application in complex tasks. Diving into the computation of FlashAttention, we prove that its optimal efficiency is determined by the rank of the attention weight matrix. Inspired by this theoretical result, this paper presents FlashBias based on the low-rank compressed sensing theory, which can provide fast-exact computation for many widely used attention biases and a fast-accurate approximation for biases in general formalizations. FlashBias can fully take advantage of the extremely optimized matrix multiplication operation in modern GPUs, achieving 1.5$\times$ speedup for Pairformer in AlphaFold 3, and over 2$\times$ speedup for attention with bias in vision and language models without loss of accuracy. Code is available at this repository: https://github.com/thuml/FlashBias.

Method: Developed a rigorous entropic-force theory based on parameter symmetries and entropic loss landscape, analyzing learning dynamics of neural networks trained with SGD and its variants.

Result: The theory explains gradient balance phenomena resembling thermal equipartition, proves the Platonic Representation Hypothesis, and reconciles sharpness- vs flatness-seeking optimization behaviors.

Conclusion: A combination of entropic forces and symmetry breaking is key to understanding emergent phenomena in deep learning.

Abstract: With the rapid discovery of emergent phenomena in deep learning and large language models, understanding their cause has become an urgent need. Here, we propose a rigorous entropic-force theory for understanding the learning dynamics of neural networks trained with stochastic gradient descent (SGD) and its variants. Building on the theory of parameter symmetries and an entropic loss landscape, we show that representation learning is crucially governed by emergent entropic forces arising from stochasticity and discrete-time updates. These forces systematically break continuous parameter symmetries and preserve discrete ones, leading to a series of gradient balance phenomena that resemble the equipartition property of thermal systems. These phenomena, in turn, (a) explain the universal alignment of neural representations between AI models and lead to a proof of the Platonic Representation Hypothesis, and (b) reconcile the seemingly contradictory observations of sharpness- and flatness-seeking behavior of deep learning optimization. Our theory and experiments demonstrate that a combination of entropic forces and symmetry breaking is key to understanding emergent phenomena in deep learning.

Method: Leverages temporal-difference learning from reinforcement learning to identify a temporal-consistency condition, then develops a novel loss function for training incremental sequence classifiers that enforces this condition.

Result: Substantial gains in data efficiency, improved predictive accuracy on text classification benchmarks, and better ability to distinguish promising from unpromising LLM generations after observing only a few tokens in math problem verification.

Conclusion: The proposed temporal-consistency based loss function effectively improves incremental sequence classification performance across multiple tasks including text classification and LLM generation verification.

Abstract: We address the problem of incremental sequence classification, where predictions are updated as new elements in the sequence are revealed. Drawing on temporal-difference learning from reinforcement learning, we identify a temporal-consistency condition that successive predictions should satisfy. We leverage this condition to develop a novel loss function for training incremental sequence classifiers. Through a concrete example, we demonstrate that optimizing this loss can offer substantial gains in data efficiency. We apply our method to text classification tasks and show that it improves predictive accuracy over competing approaches on several benchmark datasets. We further evaluate our approach on the task of verifying large language model generations for correctness in grade-school math problems. Our results show that models trained with our method are better able to distinguish promising generations from unpromising ones after observing only a few tokens.

Method: Developed XNN architecture inspired by parameter-sharing structures like Deep Sets and Graph Neural Networks. Converted existing PDE foundation models into axial neural networks and evaluated across three training scenarios: training from scratch, pretraining on multiple PDEs, and fine-tuning on a single PDE.

Result: XNNs perform competitively with original models and demonstrate superior generalization to unseen dimensions. Shows the importance of multidimensional pretraining for foundation models.

Conclusion: The XNN architecture effectively addresses the dimensionality challenge in physics foundation models, enabling efficient training and better generalization across varying dimensional PDE systems.

Abstract: The advent of foundation models in AI has significantly advanced general-purpose learning, enabling remarkable capabilities in zero-shot inference and in-context learning. However, training such models on physics data, including solutions to partial differential equations (PDEs), poses a unique challenge due to varying dimensionalities across different systems. Traditional approaches either fix a maximum dimension or employ separate encoders for different dimensionalities, resulting in inefficiencies. To address this, we propose a dimension-agnostic neural network architecture, the Axial Neural Network (XNN), inspired by parameter-sharing structures such as Deep Sets and Graph Neural Networks. XNN generalizes across varying tensor dimensions while maintaining computational efficiency. We convert existing PDE foundation models into axial neural networks and evaluate their performance across three training scenarios: training from scratch, pretraining on multiple PDEs, and fine-tuning on a single PDE. Our experiments show that XNNs perform competitively with original models and exhibit superior generalization to unseen dimensions, highlighting the importance of multidimensional pretraining for foundation models.

Method: Introduces latent calibration concept and latent recalibration (LR) - a post-hoc method that learns transformations in the latent space of conditional normalizing flows with finite-sample bounds on calibration.

Result: Extensive experiments on tabular and image datasets show LR consistently improves latent calibration error and negative log-likelihood of recalibrated models.

Conclusion: LR effectively addresses multivariate calibration while providing explicit density functions, outperforming existing methods in both calibration and likelihood metrics.

Abstract: Reliably characterizing the full conditional distribution of a multivariate response variable given a set of covariates is crucial for trustworthy decision-making. However, misspecified or miscalibrated multivariate models may yield a poor approximation of the joint distribution of the response variables, leading to unreliable predictions and suboptimal decisions. Furthermore, standard recalibration methods are primarily limited to univariate settings, while conformal prediction techniques, despite generating multivariate prediction regions with coverage guarantees, do not provide a full probability density function. We address this gap by first introducing a novel notion of latent calibration, which assesses probabilistic calibration in the latent space of a conditional normalizing flow. Second, we propose latent recalibration (LR), a novel post-hoc model recalibration method that learns a transformation of the latent space with finite-sample bounds on latent calibration. Unlike existing methods, LR produces a recalibrated distribution with an explicit multivariate density function while remaining computationally efficient. Extensive experiments on both tabular and image datasets show that LR consistently improves latent calibration error and the negative log-likelihood of the recalibrated models.

Method: Proposes SAMOSA (Sharpness Aware Minimization for Open Set Active Learning) that actively queries samples based on their typicality, identifying atypical samples near model decision boundaries using theoretical insights from SGD and SAM optimization.

Result: SAMOSA achieves up to 3% accuracy improvement over state-of-the-art methods across several datasets while maintaining computational efficiency.

Conclusion: SAMOSA effectively addresses open set active learning challenges by leveraging sharpness-aware minimization for sample selection, providing both improved performance and practical efficiency.

Abstract: Modern machine learning solutions require extensive data collection where labeling remains costly. To reduce this burden, open set active learning approaches aim to select informative samples from a large pool of unlabeled data that includes irrelevant or unknown classes. In this context, we propose Sharpness Aware Minimization for Open Set Active Learning (SAMOSA) as an effective querying algorithm. Building on theoretical findings concerning the impact of data typicality on the generalization properties of traditional stochastic gradient descent (SGD) and sharpness-aware minimization (SAM), SAMOSA actively queries samples based on their typicality. SAMOSA effectively identifies atypical samples that belong to regions of the embedding manifold close to the model decision boundaries. Therefore, SAMOSA prioritizes the samples that are (i) highly informative for the targeted classes, and (ii) useful for distinguishing between targeted and unwanted classes. Extensive experiments show that SAMOSA achieves up to 3% accuracy improvement over the state of the art across several datasets, while not introducing computational overhead. The source code of our experiments is available at: https://anonymous.4open.science/r/samosa-DAF4

Method: Proposes a dimensionality compression goodness function using effective dimensionality (ED) of fluctuating neural responses to incorporate second-order statistical structure. The objective minimizes ED for noisy copies of individual inputs while maximizing it across the sample distribution.

Result: The method achieves competitive performance compared to other non-backpropagation methods. Noise plays a constructive role in enhancing generalization and improving inference when predictions are derived from mean squared output.

Conclusion: The approach contributes to more biologically plausible learning algorithms and is naturally suited for neuromorphic computing where stochasticity is a computational resource rather than a nuisance.

Abstract: The Forward-Forward (FF) learning algorithm provides a bottom-up alternative to backpropagation (BP) for training neural networks, relying on a layer-wise “goodness” function with well-designed negative samples for contrastive learning. Existing goodness functions are typically defined as the sum of squared postsynaptic activations, neglecting correlated variability between neurons. In this work, we propose a novel goodness function termed dimensionality compression that uses the effective dimensionality (ED) of fluctuating neural responses to incorporate second-order statistical structure. Our objective minimizes ED for noisy copies of individual inputs while maximizing it across the sample distribution, promoting structured representations without the need to prepare negative samples.We demonstrate that this formulation achieves competitive performance compared to other non-BP methods. Moreover, we show that noise plays a constructive role that can enhance generalization and improve inference when predictions are derived from the mean of squared output, which is equivalent to making predictions based on an energy term. Our findings contribute to the development of more biologically plausible learning algorithms and suggest a natural fit for neuromorphic computing, where stochasticity is a computational resource rather than a nuisance. The code is available at https://github.com/ZhichaoZhu/StochasticForwardForward

Method: Proposes STAR-DSS framework that repurposes guardrail models to evaluate partial responses and generate Safety Trajectory Assessment of Response (STAR) - token-level safety signals that enable dynamic shaping during training sequences.

Result: STAR-DSS robustly mitigates finetuning risks and delivers substantial safety improvements across diverse threats, datasets, and model families without compromising capability on intended tasks.

Conclusion: Dynamic safety shaping principles provide stronger mitigation against evolving finetuning risks compared to static approaches, and future safety research should build on these dynamic shaping methods.

Abstract: Finetuning large language models (LLMs) enables user-specific customization but introduces critical safety risks: even a few harmful examples can compromise safety alignment. A common mitigation strategy is to update the model more strongly on examples deemed safe, while downweighting or excluding those flagged as unsafe. However, because safety context can shift within a single example, updating the model equally on both harmful and harmless parts of a response is suboptimal-a coarse treatment we term static safety shaping. In contrast, we propose dynamic safety shaping (DSS), a framework that uses fine-grained safety signals to reinforce learning from safe segments of a response while suppressing unsafe content. To enable such fine-grained control during finetuning, we introduce a key insight: guardrail models, traditionally used for filtering, can be repurposed to evaluate partial responses, tracking how safety risk evolves throughout the response, segment by segment. This leads to the Safety Trajectory Assessment of Response (STAR), a token-level signal that enables shaping to operate dynamically over the training sequence. Building on this, we present STAR-DSS, guided by STAR scores, that robustly mitigates finetuning risks and delivers substantial safety improvements across diverse threats, datasets, and model families-all without compromising capability on intended tasks. We encourage future safety research to build on dynamic shaping principles for stronger mitigation against evolving finetuning risks. Our code is publicly available at https://github.com/poloclub/star-dss.

Method: GRADATE leverages optimal transport theory to select the best training samples from source domains for target domain classification, operating without GNN model predictions or specialized training procedures.

Result: GRADATE outperforms existing selection methods and enhances off-the-shelf GDA approaches using significantly less training data across multiple real-world graph datasets and covariate shift types.

Conclusion: GRADATE provides a scalable, data-efficient framework that complements model-centric GDA methods and effectively addresses distribution shift challenges in graph machine learning.

Abstract: Graph domain adaptation (GDA) is a fundamental task in graph machine learning, with techniques like shift-robust graph neural networks (GNNs) and specialized training procedures to tackle the distribution shift problem. Although these model-centric approaches show promising results, they often struggle with severe shifts and constrained computational resources. To address these challenges, we propose a novel model-free framework, GRADATE (GRAph DATa sElector), that selects the best training data from the source domain for the classification task on the target domain. GRADATE picks training samples without relying on any GNN model’s predictions or training recipes, leveraging optimal transport theory to capture and adapt to distribution changes. GRADATE is data-efficient, scalable and meanwhile complements existing model-centric GDA approaches. Through comprehensive empirical studies on several real-world graph-level datasets and multiple covariate shift types, we demonstrate that GRADATE outperforms existing selection methods and enhances off-the-shelf GDA methods with much fewer training data.

Method: The authors formalize the selective-classification gap and provide the first finite-sample decomposition into five distinct error sources. They validate this decomposition through controlled experiments on synthetic two-moons data and real-world vision and language benchmarks.

Result: The analysis reveals that: (i) Bayes noise and limited model capacity account for substantial gaps, (ii) only feature-aware calibrators meaningfully improve score ordering, and (iii) data shift introduces separate slack requiring distributionally robust training.

Conclusion: The decomposition provides a quantitative error budget and actionable design guidelines for practitioners to build selective classifiers that more closely approximate ideal oracle behavior, emphasizing the need for scoring mechanisms that can effectively reorder predictions rather than just rescale them.

Abstract: Selective classifiers improve model reliability by abstaining on inputs the model deems uncertain. However, few practical approaches achieve the gold-standard performance of a perfect-ordering oracle that accepts examples exactly in order of correctness. Our work formalizes this shortfall as the selective-classification gap and present the first finite-sample decomposition of this gap to five distinct sources of looseness: Bayes noise, approximation error, ranking error, statistical noise, and implementation- or shift-induced slack. Crucially, our analysis reveals that monotone post-hoc calibration – often believed to strengthen selective classifiers – has limited impact on closing this gap, since it rarely alters the model’s underlying score ranking. Bridging the gap therefore requires scoring mechanisms that can effectively reorder predictions rather than merely rescale them. We validate our decomposition on synthetic two-moons data and on real-world vision and language benchmarks, isolating each error component through controlled experiments. Our results confirm that (i) Bayes noise and limited model capacity can account for substantial gaps, (ii) only richer, feature-aware calibrators meaningfully improve score ordering, and (iii) data shift introduces a separate slack that demands distributionally robust training. Together, our decomposition yields a quantitative error budget as well as actionable design guidelines that practitioners can use to build selective classifiers which approximate ideal oracle behavior more closely.

Method: Uses structured linear controlled differential equations with input-dependent state-transition matrices including block-diagonal, sparse, and Walsh-Hadamard variants, while maintaining dense matrix expressivity.

Result: SLiCEs solve the A5 state-tracking benchmark with one layer, achieve best length generalization on regular language tasks, and match log neural CDE performance on time-series classification with 20x faster training.

Conclusion: SLiCEs provide a unifying framework that combines computational efficiency with maximal expressivity, outperforming existing sequence models across multiple benchmarks.

Abstract: This work introduces Structured Linear Controlled Differential Equations (SLiCEs), a unifying framework for sequence models with structured, input-dependent state-transition matrices that retain the maximal expressivity of dense matrices whilst being cheaper to compute. The framework encompasses existing architectures, such as input-dependent block-diagonal linear recurrent neural networks and DeltaNet’s diagonal-plus-low-rank structure, as well as two novel variants based on sparsity and the Walsh-Hadamard transform. We prove that, unlike the diagonal state-transition matrices of S4D and Mamba, SLiCEs employing block-diagonal, sparse, or Walsh-Hadamard matrices match the maximal expressivity of dense matrices. Empirically, SLiCEs solve the $A_5$ state-tracking benchmark with a single layer, achieve best-in-class length generalisation on regular language tasks among parallel-in-time models, and match the performance of log neural controlled differential equations on six multivariate time-series classification datasets while cutting the average time per training step by a factor of twenty.

Method: Uses pullback metrics induced by global diffeomorphisms to make Riemannian CFM equivalent to standard CFM after data transformation, with instantiations using matrix logarithm for covariance matrices and normalized Cholesky decomposition for correlation matrices.

Result: Achieves state-of-the-art performance on large-scale fMRI datasets (4600+ scans from 2800 subjects) and EEG motor imagery datasets (30000+ trials from 26 subjects) with fast training while preserving manifold constraints.

Conclusion: DiffeoCFM provides an efficient framework for generative modeling on matrix manifolds that enables fast training and sampling while maintaining geometric constraints, with applications in brain connectivity analysis.

Abstract: Generating realistic brain connectivity matrices is key to analyzing population heterogeneity in brain organization, understanding disease, and augmenting data in challenging classification problems. Functional connectivity matrices lie in constrained spaces, such as the set of symmetric positive definite or correlation matrices, that can be modeled as Riemannian manifolds. However, using Riemannian tools typically requires redefining core operations (geodesics, norms, integration), making generative modeling computationally inefficient. In this work, we propose DiffeoCFM, an approach that enables conditional flow matching (CFM) on matrix manifolds by exploiting pullback metrics induced by global diffeomorphisms on Euclidean spaces. We show that Riemannian CFM with such metrics is equivalent to applying standard CFM after data transformation. This equivalence allows efficient vector field learning, and fast sampling with standard ODE solvers. We instantiate DiffeoCFM with two different settings: the matrix logarithm for covariance matrices and the normalized Cholesky decomposition for correlation matrices. We evaluate DiffeoCFM on three large-scale fMRI datasets with more than 4600 scans from 2800 subjects (ADNI, ABIDE, OASIS-3) and two EEG motor imagery datasets with over 30000 trials from 26 subjects (BNCI2014-002 and BNCI2015-001). It enables fast training and achieves state-of-the-art performance, all while preserving manifold constraints. Code: https://github.com/antoinecollas/DiffeoCFM

Method: Constructs a joint graph integrating atomic structures of drug pairs, uses structure consistency module to iteratively refine node features while preserving global structural context, and models both local and global interaction patterns.

Result: Outperforms state-of-the-art baselines, achieves superior performance across long-tail and inductive scenarios, and demonstrates robust performance across both frequent and rare DDI types on two benchmark datasets.

Conclusion: Fine-grained graph refinement improves accuracy, robustness, and mechanistic interpretability of DDI event prediction, contributing to graph-based methods for mining drug-drug interaction networks.

Abstract: Drug combinations offer therapeutic benefits but also carry the risk of adverse drug-drug interactions (DDIs), especially under complex molecular structures. Accurate DDI event prediction requires capturing fine-grained inter-drug relationships, which are critical for modeling metabolic mechanisms such as enzyme-mediated competition. However, existing approaches typically rely on isolated drug representations and fail to explicitly model atom-level cross-molecular interactions, limiting their effectiveness across diverse molecular complexities and DDI type distributions. To address these limitations, we propose MolBridge, a novel atom-level joint graph refinement framework for robust DDI event prediction. MolBridge constructs a joint graph that integrates atomic structures of drug pairs, enabling direct modeling of inter-drug associations. A central challenge in such joint graph settings is the potential loss of information caused by over-smoothing when modeling long-range atomic dependencies. To overcome this, we introduce a structure consistency module that iteratively refines node features while preserving the global structural context. This joint design allows MolBridge to effectively learn both local and global interaction outperforms state-of-the-art baselines, achieving superior performance across long-tail and inductive scenarios. patterns, yielding robust representations across both frequent and rare DDI types. Extensive experiments on two benchmark datasets show that MolBridge consistently. These results demonstrate the advantages of fine-grained graph refinement in improving the accuracy, robustness, and mechanistic interpretability of DDI event prediction.This work contributes to Web Mining and Content Analysis by developing graph-based methods for mining and analyzing drug-drug interaction networks.

Method: Proposed new algorithms for both non-contextual and contextual settings that learn reward distributions and linear functions while making sequential box-opening decisions with semi-bandit feedback.

Result: Achieved O(√nT) regret for non-contextual setting (matching lower bound) and O(nd√T) regret for contextual linear setting, significantly improving previous bounds.

Conclusion: The proposed algorithms provide optimal regret bounds for online Pandora’s Box problems and can be extended to related problems like Prophet Inequality, demonstrating broad applicability of the techniques.

Abstract: We study the Pandora’s Box problem in an online learning setting with semi-bandit feedback. In each round, the learner sequentially pays to open up to $n$ boxes with unknown reward distributions, observes rewards upon opening, and decides when to stop. The utility of the learner is the maximum observed reward minus the cumulative cost of opened boxes, and the goal is to minimize regret defined as the gap between the cumulative expected utility and that of the optimal policy. We propose a new algorithm that achieves $\widetilde{O}(\sqrt{nT})$ regret after $T$ rounds, which improves the $\widetilde{O}(n\sqrt{T})$ bound of Agarwal et al. [2024] and matches the known lower bound up to logarithmic factors. To better capture real-life applications, we then extend our results to a natural but challenging contextual linear setting, where each box’s expected reward is linear in some known but time-varying $d$-dimensional context and the noise distribution is fixed over time. We design an algorithm that learns both the linear function and the noise distributions, achieving $\widetilde{O}(nd\sqrt{T})$ regret. Finally, we show that our techniques also apply to the online Prophet Inequality problem, where the learner must decide immediately whether or not to accept a revealed reward. In both non-contextual and contextual settings, our approach achieves similar improvements and regret bounds.

Method: Uses dynamical systems analysis with Jacobian matrices to study self-attention layers, focusing on how normalization affects Lipschitzness, complex eigenvalues, and critical states. Also develops regularization methods and pseudo-energy monitoring.

Result: Reveals that normalization layers suppress SA Lipschitzness and Jacobian complex eigenvalues (oscillatory components), and that normalized dynamics lie near critical states which strongly indicate high inference performance.

Conclusion: The Jacobian perspective provides valuable insights into self-attention dynamics without energy constraints, enables development of regularization methods, and shows that criticality serves as a key indicator of inference performance.

Abstract: The theoretical understanding of self-attention (SA) has been steadily progressing. A prominent line of work studies a class of SA layers that admit an energy function decreased by state updates. While it provides valuable insights into inherent biases in signal propagation, it often relies on idealized assumptions or additional constraints not necessarily present in standard SA. Thus, to broaden our understanding, this work aims to relax these energy constraints and provide an energy-agnostic characterization of inference dynamics by dynamical systems analysis. In more detail, we first consider relaxing the symmetry and single-head constraints traditionally required in energy-based formulations. Next, we show that analyzing the Jacobian matrix of the state is highly valuable when investigating more general SA architectures without necessarily admitting an energy function. It reveals that the normalization layer plays an essential role in suppressing the Lipschitzness of SA and the Jacobian’s complex eigenvalues, which correspond to the oscillatory components of the dynamics. In addition, the Lyapunov exponents computed from the Jacobians demonstrate that the normalized dynamics lie close to a critical state, and this criticality serves as a strong indicator of high inference performance. Furthermore, the Jacobian perspective also enables us to develop regularization methods for training and a pseudo-energy for monitoring inference dynamics.

Method: The approach relies on a norm-boundedness assumption on the unknown function and noise, computing worst-case function realizations using Gaussian process posterior mean and covariance with optimal measurement noise covariance selection.

Result: The method provides tight and easy-to-compute uncertainty bounds for kernel-based estimates, showing effectiveness through rigorous analysis and comparison with existing literature.

Conclusion: The proposed approach successfully delivers non-asymptotic, tight uncertainty bounds for kernel-based estimation that can handle correlated noise, making it valuable for safety-critical applications requiring reliable uncertainty quantification.

Abstract: Non-conservative uncertainty bounds are key for both assessing an estimation algorithm’s accuracy and in view of downstream tasks, such as its deployment in safety-critical contexts. In this paper, we derive a tight, non-asymptotic uncertainty bound for kernel-based estimation, which can also handle correlated noise sequences. Its computation relies on a mild norm-boundedness assumption on the unknown function and the noise, returning the worst-case function realization within the hypothesis class at an arbitrary query input location. The value of this function is shown to be given in terms of the posterior mean and covariance of a Gaussian process for an optimal choice of the measurement noise covariance. By rigorously analyzing the proposed approach and comparing it with other results in the literature, we show its effectiveness in returning tight and easy-to-compute bounds for kernel-based estimates.

Method: Developed Neyman-orthogonal loss functions incorporating response time data using the Evidence Accumulation Drift Diffusion (EZ) model, achieving oracle convergence rates for reward model learning.

Result: The response time-augmented approach reduces error rates from exponential to polynomial scaling with reward magnitude, significantly improving sample efficiency. Theoretical guarantees extend to non-parametric reward functions.

Conclusion: Incorporating response time data alongside binary preferences substantially improves reward model learning efficiency and performance, with validated results in image preference learning experiments.

Abstract: This paper investigates the integration of response time data into human preference learning frameworks for more effective reward model elicitation. While binary preference data has become fundamental in fine-tuning foundation models, generative AI systems, and other large-scale models, the valuable temporal information inherent in user decision-making remains largely unexploited. We propose novel methodologies to incorporate response time information alongside binary choice data, leveraging the Evidence Accumulation Drift Diffusion (EZ) model, under which response time is informative of the preference strength. We develop Neyman-orthogonal loss functions that achieve oracle convergence rates for reward model learning, matching the theoretical optimal rates that would be attained if the expected response times for each query were known a priori. Our theoretical analysis demonstrates that for linear reward functions, conventional preference learning suffers from error rates that scale exponentially with reward magnitude. In contrast, our response time-augmented approach reduces this to polynomial scaling, representing a significant improvement in sample efficiency. We extend these guarantees to non-parametric reward function spaces, establishing convergence properties for more complex, realistic reward models. Our extensive experiments validate our theoretical findings in the context of preference learning over images.

Method: Tightly integrates DNA foundation model with LLM through supervised fine-tuning and reinforcement learning to enable direct interpretation and reasoning over genomic information.

Result: Boosts KEGG-based disease pathway prediction accuracy from 86% to 98% and improves variant effect prediction by average 15% over baselines. Can reason over unseen biological entities with step-by-step explanations.

Conclusion: BioReason offers a transformative framework for interpretable, mechanistic AI in biology, enabling logical and biologically coherent deductions.

Abstract: Unlocking deep and interpretable biological reasoning from complex genomic data remains a major AI challenge limiting scientific progress. While current DNA foundation models excel at representing sequences, they struggle with multi-step reasoning and lack transparent, biologically meaningful explanations. BioReason addresses this by tightly integrating a DNA foundation model with a large language model (LLM), enabling the LLM to directly interpret and reason over genomic information. Through supervised fine-tuning and reinforcement learning, BioReason learns to produce logical, biologically coherent deductions. It achieves major performance gains, boosting KEGG-based disease pathway prediction accuracy from 86% to 98% and improving variant effect prediction by an average of 15% over strong baselines. BioReason can reason over unseen biological entities and explain its decisions step by step, offering a transformative framework for interpretable, mechanistic AI in biology. All data, code, and checkpoints are available at https://github.com/bowang-lab/BioReason

Method: Introduce a framework where transferability corresponds to continuity in a limit space, identify small and large problem instances as equivalent based on data and task, and implement necessary changes to existing architectures.

Result: The framework provides design principles for transferable models, and numerical experiments support the findings.

Conclusion: Transferability across dimensions can be systematically achieved through continuity in a properly defined limit space, enabling performance transfer from small to large problem instances.

Abstract: Many modern learning tasks require models that can take inputs of varying sizes. Consequently, dimension-independent architectures have been proposed for domains where the inputs are graphs, sets, and point clouds. Recent work on graph neural networks has explored whether a model trained on low-dimensional data can transfer its performance to higher-dimensional inputs. We extend this body of work by introducing a general framework for transferability across dimensions. We show that transferability corresponds precisely to continuity in a limit space formed by identifying small problem instances with equivalent large ones. This identification is driven by the data and the learning task. We instantiate our framework on existing architectures, and implement the necessary changes to ensure their transferability. Finally, we provide design principles for designing new transferable models. Numerical experiments support our findings.

Method: Analyzed two-layer ReLU networks on binary classification with Gaussian data, comparing population gradients and conducting extensive empirical validation across datasets with spurious correlations.

Result: GD produces linear decision boundaries with suboptimal margins, while Adam creates nonlinear boundaries closer to Bayes’ optimal predictor, achieving higher test accuracy in-distribution and under distribution shifts.

Conclusion: Adam’s resistance to simplicity bias enables richer feature learning and superior generalization compared to SGD, especially in the presence of spurious correlations and distributional shifts.

Abstract: Adam is the de facto optimization algorithm for several deep learning applications, but an understanding of its implicit bias and how it differs from other algorithms, particularly standard first-order methods such as (stochastic) gradient descent (GD), remains limited. In practice, neural networks (NNs) trained with SGD are known to exhibit simplicity bias – a tendency to find simple solutions. In contrast, we show that Adam is more resistant to such simplicity bias. First, we investigate the differences in the implicit biases of Adam and GD when training two-layer ReLU NNs on a binary classification task with Gaussian data. We find that GD exhibits a simplicity bias, resulting in a linear decision boundary with a suboptimal margin, whereas Adam leads to much richer and more diverse features, producing a nonlinear boundary that is closer to the Bayes’ optimal predictor. This richer decision boundary also allows Adam to achieve higher test accuracy both in-distribution and under certain distribution shifts. We theoretically prove these results by analyzing the population gradients. Next, to corroborate our theoretical findings, we present extensive empirical results showing that this property of Adam leads to superior generalization across various datasets with spurious correlations where NNs trained with SGD are known to show simplicity bias and do not generalize well under certain distributional shifts.

Method: Proposes Feasibility-Seeking Neural Network (FSNet) that incorporates a differentiable feasibility-seeking step to minimize constraint violations, enabling end-to-end training with guarantees on feasibility and convergence.

Result: Experiments show FSNet provides feasible solutions with quality comparable to or better than traditional solvers across various optimization problems (smooth/nonsmooth, convex/nonconvex) at significantly faster speeds.

Conclusion: FSNet effectively bridges the gap between computational efficiency and constraint satisfaction in optimization, offering a practical solution for real-time applications.

Abstract: Efficiently solving constrained optimization problems is crucial for numerous real-world applications, yet traditional solvers are often computationally prohibitive for real-time use. Machine learning-based approaches have emerged as a promising alternative to provide approximate solutions at faster speeds, but they struggle to strictly enforce constraints, leading to infeasible solutions in practice. To address this, we propose the Feasibility-Seeking Neural Network (FSNet), which integrates a feasibility-seeking step directly into its solution procedure to ensure constraint satisfaction. This feasibility-seeking step solves an unconstrained optimization problem that minimizes constraint violations in a differentiable manner, enabling end-to-end training and providing guarantees on feasibility and convergence. Our experiments across a range of different optimization problems, including both smooth/nonsmooth and convex/nonconvex problems, demonstrate that FSNet can provide feasible solutions with solution quality comparable to (or in some cases better than) traditional solvers, at significantly faster speeds.

Method: Uses a graph attention network that operates in self-supervised manner, eliminating need for labeled training data. The approach specifically handles pilot contamination and can adapt to dynamic numbers of user equipments without retraining.

Result: Experimental results demonstrate the method’s effectiveness, showing comparable or better performance than the optimal accelerated projected gradient method while being computationally efficient for real-time applications.

Conclusion: The proposed self-supervised graph attention network provides a practical solution for real-time power control in CFmMIMO systems, addressing key limitations of existing methods including pilot contamination, dynamic UE numbers, and computational complexity.

Abstract: Optimization-based power control algorithms are predominantly iterative with high computational complexity, making them impractical for real-time applications in cell-free massive multiple-input multiple-output (CFmMIMO) systems. Learning-based methods have emerged as a promising alternative, and among them, graph neural networks (GNNs) have demonstrated their excellent performance in solving power control problems. However, all existing GNN-based approaches assume ideal orthogonality among pilot sequences for user equipments (UEs), which is unrealistic given that the number of UEs exceeds the available orthogonal pilot sequences in CFmMIMO schemes. Moreover, most learning-based methods assume a fixed number of UEs, whereas the number of active UEs varies over time in practice. Additionally, supervised training necessitates costly computational resources for computing the target power control solutions for a large volume of training samples. To address these issues, we propose a graph attention network for downlink power control in CFmMIMO systems that operates in a self-supervised manner while effectively handling pilot contamination and adapting to a dynamic number of UEs. Experimental results show its effectiveness, even in comparison to the optimal accelerated projected gradient method as a baseline.

Method: The authors develop adaptive algorithms that do not require prior knowledge about the range or second moments of losses. The algorithms are designed to work with only an upper bound on the second moments of losses.

Result: The proposed algorithms guarantee O(√(θTlog(K))) regret in worst-case scenarios and O(θlog(KT)/Δ_min) regret when losses are i.i.d. from a fixed distribution, where Δ_min is the gap between the best and second-best experts. For squared loss, the algorithms also achieve improved regret bounds over prior work.

Conclusion: The paper successfully addresses the problem of heavy-tailed losses in prediction with expert advice by developing adaptive algorithms that eliminate problematic lower-order terms and provide improved regret guarantees across various scenarios.

Abstract: We consider the problem setting of prediction with expert advice with possibly heavy-tailed losses, i.e.\ the only assumption on the losses is an upper bound on their second moments, denoted by $\theta$. We develop adaptive algorithms that do not require any prior knowledge about the range or the second moment of the losses. Existing adaptive algorithms have what is typically considered a lower-order term in their regret guarantees. We show that this lower-order term, which is often the maximum of the losses, can actually dominate the regret bound in our setting. Specifically, we show that even with small constant $\theta$, this lower-order term can scale as $\sqrt{KT}$, where $K$ is the number of experts and $T$ is the time horizon. We propose adaptive algorithms with improved regret bounds that avoid the dependence on such a lower-order term and guarantee $\mathcal{O}(\sqrt{\theta T\log(K)})$ regret in the worst case, and $\mathcal{O}(\theta \log(KT)/\Delta_{\min})$ regret when the losses are sampled i.i.d.\ from some fixed distribution, where $\Delta_{\min}$ is the difference between the mean losses of the second best expert and the best expert. Additionally, when the loss function is the squared loss, our algorithm also guarantees improved regret bounds over prior results.

Method: Uses Kalman-based optimization with recursive updates of compact gradient covariance products to estimate parameter covariance matrix, avoiding full covariance storage and large matrix inversions.

Result: Achieves accuracy on par or better than state-of-the-art first- and second-order optimizers across diverse tasks including image classification and language modeling, while maintaining first-order method efficiency.

Conclusion: KOALA++ successfully bridges the gap between first-order efficiency and second-order accuracy by modeling structured gradient uncertainty through compact covariance estimation.

Abstract: We propose KOALA++, a scalable Kalman-based optimization algorithm that explicitly models structured gradient uncertainty in neural network training. Unlike second-order methods, which rely on expensive second order gradient calculation, our method directly estimates the parameter covariance matrix by recursively updating compact gradient covariance products. This design improves upon the original KOALA framework that assumed diagonal covariance by implicitly capturing richer uncertainty structure without storing the full covariance matrix and avoiding large matrix inversions. Across diverse tasks, including image classification and language modeling, KOALA++ achieves accuracy on par or better than state-of-the-art first- and second-order optimizers while maintaining the efficiency of first-order methods.

Method: Learning to Search (L2S) approach that learns a world model and reward model from demonstrations, enabling planning at test time to recover from mistakes without additional human corrections.

Result: SAILOR consistently outperforms state-of-the-art Diffusion Policies trained via BC across 12 visual manipulation tasks, maintaining performance advantage even when BC uses 5-10x more demonstrations. The method identifies nuanced failures and is robust to reward hacking.

Conclusion: Learning to search from demonstrations provides superior recovery capabilities compared to behavioral cloning, enabling agents to plan and achieve expert outcomes even after making mistakes in unseen situations.

Abstract: The fundamental limitation of the behavioral cloning (BC) approach to imitation learning is that it only teaches an agent what the expert did at states the expert visited. This means that when a BC agent makes a mistake which takes them out of the support of the demonstrations, they often don’t know how to recover from it. In this sense, BC is akin to giving the agent the fish – giving them dense supervision across a narrow set of states – rather than teaching them to fish: to be able to reason independently about achieving the expert’s outcome even when faced with unseen situations at test-time. In response, we explore learning to search (L2S) from expert demonstrations, i.e. learning the components required to, at test time, plan to match expert outcomes, even after making a mistake. These include (1) a world model and (2) a reward model. We carefully ablate the set of algorithmic and design decisions required to combine these and other components for stable and sample/interaction-efficient learning of recovery behavior without additional human corrections. Across a dozen visual manipulation tasks from three benchmarks, our approach SAILOR consistently out-performs state-of-the-art Diffusion Policies trained via BC on the same data. Furthermore, scaling up the amount of demonstrations used for BC by 5-10x still leaves a performance gap. We find that SAILOR can identify nuanced failures and is robust to reward hacking. Our code is available at https://github.com/arnavkj1995/SAILOR .

Method: Modified score network architecture to compute energy while preserving inductive biases, trained with dual score matching objective that optimizes both gradient with respect to input image and noise level.

Result: Achieved cross-entropy comparable to state-of-the-art on ImageNet64, showed strong generalization across non-overlapping data subsets, and revealed that image probability and local dimensionality vary substantially with content.

Conclusion: The proposed energy modeling framework successfully addresses normalization challenges and provides insights into the complex nature of image distributions, challenging conventional assumptions about concentration of measure and low-dimensional manifolds.

Abstract: Learning probability models from data is at the heart of many machine learning endeavors, but is notoriously difficult due to the curse of dimensionality. We introduce a new framework for learning \emph{normalized} energy (log probability) models that is inspired from diffusion generative models, which rely on networks optimized to estimate the score. We modify a score network architecture to compute an energy while preserving its inductive biases. The gradient of this energy network with respect to its input image is the score of the learned density, which can be optimized using a denoising objective. Importantly, the gradient with respect to the noise level provides an additional score that can be optimized with a novel secondary objective, ensuring consistent and normalized energies across noise levels. We train an energy network with this \emph{dual} score matching objective on the ImageNet64 dataset, and obtain a cross-entropy (negative log likelihood) value comparable to the state of the art. We further validate our approach by showing that our energy model \emph{strongly generalizes}: log probabilities estimated with two networks trained on non-overlapping data subsets are nearly identical. Finally, we demonstrate that both image probability and dimensionality of local neighborhoods vary substantially depending on image content, in contrast with conventional assumptions such as concentration of measure or support on a low-dimensional manifold.

Method: Alternating Gradient Flows (AGF) approximates training dynamics as a two-step process: maximizing utility over dormant neurons and minimizing cost over active neurons. It starts with all neurons dormant and activates one per iteration, triggering feature acquisition and loss drops.

Result: AGF accurately predicts the order, timing, and magnitude of loss drops in experiments across various architectures. It unifies existing analyses in linear networks and transformers, and provides the first complete characterization of training dynamics in quadratic networks for modular addition, revealing Fourier feature learning by coefficient magnitude.

Conclusion: AGF offers a promising framework for understanding feature learning dynamics in neural networks, explaining the staircase loss pattern and providing insights into how networks progressively acquire features during training.

Abstract: What features neural networks learn, and how, remains an open question. In this paper, we introduce Alternating Gradient Flows (AGF), an algorithmic framework that describes the dynamics of feature learning in two-layer networks trained from small initialization. Prior works have shown that gradient flow in this regime exhibits a staircase-like loss curve, alternating between plateaus where neurons slowly align to useful directions and sharp drops where neurons rapidly grow in norm. AGF approximates this behavior as an alternating two-step process: maximizing a utility function over dormant neurons and minimizing a cost function over active ones. AGF begins with all neurons dormant. At each iteration, a dormant neuron activates, triggering the acquisition of a feature and a drop in the loss. AGF quantifies the order, timing, and magnitude of these drops, matching experiments across several commonly studied architectures. We show that AGF unifies and extends existing saddle-to-saddle analyses in fully connected linear networks and attention-only linear transformers, where the learned features are singular modes and principal components, respectively. In diagonal linear networks, we prove AGF converges to gradient flow in the limit of vanishing initialization. Applying AGF to quadratic networks trained to perform modular addition, we give the first complete characterization of the training dynamics, revealing that networks learn Fourier features in decreasing order of coefficient magnitude. Altogether, AGF offers a promising step towards understanding feature learning in neural networks.

Method: Proposes SPlus with three improvements: 1) bounded updates combining historical eigenbasis with instantaneous normalization for stability, 2) shape-aware scaling for learning rate transfer across network width, 3) iterate-averaging to reduce parameter noise from high learning rates.

Result: On Transformer training benchmarks across language modeling, image classification, and diffusion modeling, SPlus reaches Adam’s validation performance with 44-58% fewer gradient steps and 62-83% less wallclock time.

Conclusion: SPlus successfully addresses key practical limitations of Shampoo optimizers, providing stable, efficient optimization that significantly outperforms Adam in both computational efficiency and training speed across diverse deep learning tasks.

Abstract: In this work, we take an experimentally grounded look at neural network optimization. Building on the Shampoo family of algorithms, we identify and alleviate three key issues, resulting in the proposed SPlus method. First, we find that naive Shampoo is prone to divergence when matrix-inverses are cached for long periods. We introduce an alternate bounded update combining a historical eigenbasis with instantaneous normalization, resulting in across-the-board stability and significantly lower computational requirements. Second, we adapt a shape-aware scaling to enable learning rate transfer across network width. Third, we find that high learning rates result in large parameter noise, and propose a simple iterate-averaging scheme which unblocks faster learning. To properly confirm these findings, we introduce a pointed Transformer training benchmark, considering three objectives (language modelling, image classification, and diffusion modelling) across different stages of training. On average, SPlus is able to reach the validation performance of Adam within 44-58% of the gradient steps and 62-83% of the wallclock time.

Method: Revisit ChebNet and identify its polynomial expansion instability. Propose Stable-ChebNet by casting ChebNet as a stable, non-dissipative dynamical system without requiring eigendecompositions, positional encodings, or graph rewiring.

Result: ChebNet shows competitive advantages on long-range benchmarks compared to MPNNs and GTs. Stable-ChebNet achieves near state-of-the-art performance across several benchmarks while maintaining stability and scalability.

Conclusion: ChebNet’s spectral approach remains relevant for long-range graph modeling, and Stable-ChebNet provides a stable, efficient alternative to current methods without compromising graph structure awareness.

Abstract: ChebNet, one of the earliest spectral GNNs, has largely been overshadowed by Message Passing Neural Networks (MPNNs), which gained popularity for their simplicity and effectiveness in capturing local graph structure. Despite their success, MPNNs are limited in their ability to capture long-range dependencies between nodes. This has led researchers to adapt MPNNs through rewiring or make use of Graph Transformers, which compromises the computational efficiency that characterized early spatial message-passing architectures, and typically disregards the graph structure. Almost a decade after its original introduction, we revisit ChebNet to shed light on its ability to model distant node interactions. We find that out-of-box, ChebNet already shows competitive advantages relative to classical MPNNs and GTs on long-range benchmarks, while maintaining good scalability properties for high-order polynomials. However, we uncover that this polynomial expansion leads ChebNet to an unstable regime during training. To address this limitation, we cast ChebNet as a stable and non-dissipative dynamical system, which we coin Stable-ChebNet. Our Stable-ChebNet model allows for stable information propagation, and has controllable dynamics which do not require the use of eigendecompositions, positional encodings, or graph rewiring. Across several benchmarks, Stable-ChebNet achieves near state-of-the-art performance.

Method: Proposes Gravityformer framework that estimates spatial mass parameters, models spatial interaction using adaptive gravity model, and uses learned interactions to guide transformer attention. Includes parallel spatiotemporal graph convolution transformer.

Result: Demonstrates superior performance on six real-world datasets, with interpretable gravity attention matrix and improved generalization in cross-region inference.

Conclusion: Provides novel approach for integrating physical laws with deep learning for spatiotemporal prediction, offering interpretability and better generalization.

Abstract: Human activity intensity prediction is crucial to many location-based services. Despite tremendous progress in modeling dynamics of human activity, most existing methods overlook physical constraints of spatial interaction, leading to uninterpretable spatial correlations and over-smoothing phenomenon. To address these limitations, this work proposes a physics-informed deep learning framework, namely Gravity-informed Spatiotemporal Transformer (Gravityformer) by integrating the universal law of gravitation to refine transformer attention. Specifically, it (1) estimates two spatially explicit mass parameters based on spatiotemporal embedding feature, (2) models the spatial interaction in end-to-end neural network using proposed adaptive gravity model to learn the physical constraint, and (3) utilizes the learned spatial interaction to guide and mitigate the over-smoothing phenomenon in transformer attention. Moreover, a parallel spatiotemporal graph convolution transformer is proposed for achieving a balance between coupled spatial and temporal learning. Systematic experiments on six real-world large-scale activity datasets demonstrate the quantitative and qualitative superiority of our model over state-of-the-art benchmarks. Additionally, the learned gravity attention matrix can be not only disentangled and interpreted based on geographical laws, but also improved the generalization in zero-shot cross-region inference. This work provides a novel insight into integrating physical laws with deep learning for spatiotemporal prediction.

Method: Model the problem in the PAC framework and introduce domain shattering dimension as a new combinatorial measure to characterize domain sample complexity.

Result: The domain shattering dimension characterizes domain sample complexity and has a tight quantitative relationship with classic VC dimension.

Conclusion: Every hypothesis class learnable in standard PAC setting is also learnable in the domain generalization setting using the domain shattering dimension framework.

Abstract: We study a fundamental question of domain generalization: given a family of domains (i.e., data distributions), how many randomly sampled domains do we need to collect data from in order to learn a model that performs reasonably well on every seen and unseen domain in the family? We model this problem in the PAC framework and introduce a new combinatorial measure, which we call the domain shattering dimension. We show that this dimension characterizes the domain sample complexity. Furthermore, we establish a tight quantitative relationship between the domain shattering dimension and the classic VC dimension, demonstrating that every hypothesis class that is learnable in the standard PAC setting is also learnable in our setting.

Method: Proposes a Q-learning algorithm that leverages ERM’s dynamic consistency and elicitability properties to compute optimal stationary policies for total-reward ERM and EVaR objectives.

Result: Numerical results on tabular domains demonstrate quick and reliable convergence of the proposed Q-learning algorithm to the optimal risk-averse value function.

Conclusion: The proposed Q-learning approach successfully addresses the limitations of model-based methods by providing a model-free solution with strong convergence guarantees for risk-averse MDPs.

Abstract: Risk-averse total-reward Markov Decision Processes (MDPs) offer a promising framework for modeling and solving undiscounted infinite-horizon objectives. Existing model-based algorithms for risk measures like the entropic risk measure (ERM) and entropic value-at-risk (EVaR) are effective in small problems, but require full access to transition probabilities. We propose a Q-learning algorithm to compute the optimal stationary policy for total-reward ERM and EVaR objectives with strong convergence and performance guarantees. The algorithm and its optimality are made possible by ERM’s dynamic consistency and elicitability. Our numerical results on tabular domains demonstrate quick and reliable convergence of the proposed Q-learning algorithm to the optimal risk-averse value function.

Method: A unified meta-algorithmic framework operating under fixed-confidence and fixed-budget regimes, using adaptive confidence intervals with the same sample exploration strategy.

Result: Theoretical guarantees on solution correctness in both settings. Extensive empirical evaluations show outperformance over existing methods in accuracy and sample efficiency.

Conclusion: The proposed approach provides an effective solution for risk-aware decision-making in uncertain environments, demonstrating broad applicability across various scenarios.

Abstract: Decision making under uncertain environments in the maximization of expected reward while minimizing its risk is one of the ubiquitous problems in many subjects. Here, we introduce a novel problem setting in stochastic bandit optimization that jointly addresses two critical aspects of decision-making: maximizing expected reward and minimizing associated uncertainty, quantified via the mean-variance(MV) criterion. Unlike traditional bandit formulations that focus solely on expected returns, our objective is to efficiently and accurately identify the Pareto-optimal set of arms that strikes the best trade-off between expected performance and risk. We propose a unified meta-algorithmic framework capable of operating under both fixed-confidence and fixed-budget regimes, achieved through adaptive design of confidence intervals tailored to each scenario using the same sample exploration strategy. We provide theoretical guarantees on the correctness of the returned solutions in both settings. To complement this theoretical analysis, we conduct extensive empirical evaluations across synthetic benchmarks, demonstrating that our approach outperforms existing methods in terms of both accuracy and sample efficiency, highlighting its broad applicability to risk-aware decision-making tasks in uncertain environments.

Method: Scaling data and model size across different task encodings, theoretical analysis of multilayer perceptrons’ approximation capabilities, and linear decoding of task constituents from hidden activations.

Result: Compositional generalization emerges through scaling, with theoretical proof that standard MLPs can approximate compositional task families using linear neurons, and linear decodability correlates with composition success.

Conclusion: Neural networks can systematically capture compositional structure through sufficient scaling and coverage of task space, with practical implications for improving composition in models like text-to-image generators.

Abstract: Can neural networks systematically capture discrete, compositional task structure despite their continuous, distributed nature? The impressive capabilities of large-scale neural networks suggest that the answer to this question is yes. However, even for the most capable models, there are still frequent failure cases that raise doubts about their compositionality. Here, we seek to understand what it takes for a standard neural network to generalize over tasks that share compositional structure. We find that simply scaling data and model size leads to compositional generalization. We show that this holds across different task encodings as long as the training distribution sufficiently covers the task space. In line with this finding, we prove that standard multilayer perceptrons can approximate a general class of compositional task families to arbitrary precision using only a linear number of neurons with respect to the number of task modules. Finally, we uncover that if networks successfully compositionally generalize, the constituents of a task can be linearly decoded from their hidden activations. We show that this metric correlates with failures of text-to-image generation models to compose known concepts.

Method: Proposes a low-rank approximation approach that eliminates unstable linear-algebraic operations and integrates easily into modern deep learning pipelines for learning top-k singular functions of the Koopman operator.

Result: Empirical results show the learned singular subspaces are reliable and effective for downstream tasks including eigen-analysis and multi-step prediction.

Conclusion: The proposed method provides a scalable and conceptually simple approach for learning Koopman operator singular functions without numerical instability issues.

Abstract: The Koopman operator provides a principled framework for analyzing nonlinear dynamical systems through linear operator theory. Recent advances in dynamic mode decomposition (DMD) have shown that trajectory data can be used to identify dominant modes of a system in a data-driven manner. Building on this idea, deep learning methods such as VAMPnet and DPNet have been proposed to learn the leading singular subspaces of the Koopman operator. However, these methods require backpropagation through potentially numerically unstable operations on empirical second moment matrices, such as singular value decomposition and matrix inversion, during objective computation, which can introduce biased gradient estimates and hinder scalability to large systems. In this work, we propose a scalable and conceptually simple method for learning the top-$k$ singular functions of the Koopman operator for stochastic dynamical systems based on the idea of low-rank approximation. Our approach eliminates the need for unstable linear-algebraic operations and integrates easily into modern deep learning pipelines. Empirical results demonstrate that the learned singular subspaces are both reliable and effective for downstream tasks such as eigen-analysis and multi-step prediction.

Method: The authors propose using optimal transport theory to estimate coverage loss and mitigate arbitrary distribution shifts in conformal prediction, providing a principled solution that doesn’t require prior shift information.

Result: The method enables estimation of coverage loss and mitigation of arbitrary distribution shifts in conformal prediction, offering a broadly applicable solution.

Conclusion: Optimal transport provides a principled framework for handling distribution shifts in conformal prediction, overcoming limitations of existing methods that require prior knowledge about shift types.

Abstract: Conformal prediction is a distribution-free uncertainty quantification method that has gained popularity in the machine learning community due to its finite-sample guarantees and ease of use. Its most common variant, dubbed split conformal prediction, is also computationally efficient as it boils down to collecting statistics of the model predictions on some calibration data not yet seen by the model. Nonetheless, these guarantees only hold if the calibration and test data are exchangeable, a condition that is difficult to verify and often violated in practice due to so-called distribution shifts. The literature is rife with methods to mitigate the loss in coverage in this non-exchangeable setting, but these methods require some prior information on the type of distribution shift to be expected at test time. In this work, we study this problem via a new perspective, through the lens of optimal transport, and show that it is possible to estimate the loss in coverage and mitigate arbitrary distribution shifts, offering a principled and broadly applicable solution.

Method: Systematic study of FG-TS and SFG-TS across 11 real-world and synthetic benchmarks, comparing exact vs approximate posteriors from stochastic-gradient samplers, with ablations on preconditioning, bonus scale, and prior strength.

Result: FG-TS outperforms vanilla TS in linear and logistic bandits but tends to be weaker in neural bandits. Larger bonuses help with accurate posterior samples but hurt with sampling noise.

Conclusion: FG-TS and variants are competitive and easy-to-use, recommended as baselines in modern contextual-bandit benchmarks despite limitations in neural settings.

Abstract: Thompson Sampling (TS) is widely used to address the exploration/exploitation tradeoff in contextual bandits, yet recent theory shows that it does not explore aggressively enough in high-dimensional problems. Feel-Good Thompson Sampling (FG-TS) addresses this by adding an optimism bonus that biases toward high-reward models, and it achieves the asymptotically minimax-optimal regret in the linear setting when posteriors are exact. However, its performance with \emph{approximate} posteriors – common in large-scale or neural problems – has not been benchmarked. We provide the first systematic study of FG-TS and its smoothed variant (SFG-TS) across eleven real-world and synthetic benchmarks. To evaluate their robustness, we compare performance across settings with exact posteriors (linear and logistic bandits) to approximate regimes produced by fast but coarse stochastic-gradient samplers. Ablations over preconditioning, bonus scale, and prior strength reveal a trade-off: larger bonuses help when posterior samples are accurate, but hurt when sampling noise dominates. FG-TS generally outperforms vanilla TS in linear and logistic bandits, but tends to be weaker in neural bandits. Nevertheless, because FG-TS and its variants are competitive and easy-to-use, we recommend them as baselines in modern contextual-bandit benchmarks. Finally, we provide source code for all our experiments in https://github.com/SarahLiaw/ctx-bandits-mcmc-showdown.

Method: Developed a hybrid quantum-classical architecture based on variational quantum algorithms that can run on current NISQ devices without strict qubit or circuit depth requirements.

Result: QTFT successfully trained on forecasting datasets and outperformed classical TFT in both training and test loss on two different datasets.

Conclusion: Quantum computing shows promise for boosting deep learning architectures in complex machine learning tasks, as demonstrated by QTFT’s superior performance over classical TFT.

Abstract: The \textit{Temporal Fusion Transformer} (TFT), proposed by Lim \textit{et al.}, published in \textit{International Journal of Forecasting} (2021), is a state-of-the-art attention-based deep neural network architecture specifically designed for multi-horizon time series forecasting. It has demonstrated significant performance improvements over existing benchmarks. In this work, we introduce the Quantum Temporal Fusion Transformer (QTFT), a quantum-enhanced hybrid quantum-classical architecture that extends the capabilities of the classical TFT framework. The core idea of this work is inspired by the foundation studies, \textit{The Power of Quantum Neural Networks} by Amira Abbas \textit{et al.} and \textit{Quantum Vision Transformers} by El Amine Cherrat \textit{et al.}, published in \textit{ Nature Computational Science} (2021) and \textit{Quantum} (2024), respectively. A key advantage of our approach lies in its foundation on a variational quantum algorithm, enabling implementation on current noisy intermediate-scale quantum (NISQ) devices without strict requirements on the number of qubits or circuit depth. Our results demonstrate that QTFT is successfully trained on the forecasting datasets and is capable of accurately predicting future values. In particular, our experimental results on two different datasets display that the model outperforms its classical counterpart in terms of both training and test loss. These results indicate the prospect of using quantum computing to boost deep learning architectures in complex machine learning tasks.

Method: Combines momentum normalization technique with novel adaptive parameter choices to create algorithms for nonconvex-strongly-concave minimax and nonconvex-strongly-convex bilevel optimization.

Result: Achieves sharp convergence rates of O~(1/√T + √σ̄/T^{1/4}) for gradient norm in T iterations, where σ̄ is the stochastic gradient noise bound, without requiring prior knowledge of noise level.

Conclusion: Provides first adaptive and sharp convergence guarantees for stochastic hierarchical optimization, enabling automatic adaptivity in both low and high-noise regimes.

Abstract: Hierarchical optimization refers to problems with interdependent decision variables and objectives, such as minimax and bilevel formulations. While various algorithms have been proposed, existing methods and analyses lack adaptivity in stochastic optimization settings: they cannot achieve optimal convergence rates across a wide spectrum of gradient noise levels without prior knowledge of the noise magnitude. In this paper, we propose novel adaptive algorithms for two important classes of stochastic hierarchical optimization problems: nonconvex-strongly-concave minimax optimization and nonconvex-strongly-convex bilevel optimization. Our algorithms achieve sharp convergence rates of $\widetilde{O}(1/\sqrt{T} + \sqrt{\bar{\sigma}}/T^{1/4})$ in $T$ iterations for the gradient norm, where $\bar{\sigma}$ is an upper bound on the stochastic gradient noise. Notably, these rates are obtained without prior knowledge of the noise level, thereby enabling automatic adaptivity in both low and high-noise regimes. To our knowledge, this work provides the first adaptive and sharp convergence guarantees for stochastic hierarchical optimization. Our algorithm design combines the momentum normalization technique with novel adaptive parameter choices. Extensive experiments on synthetic and deep learning tasks demonstrate the effectiveness of our proposed algorithms.

Method: Identifies direction of distribution shift between correctly/incorrectly classified samples through lightweight offline process, then exploits this direction to universally steer all generated images toward SID failure modes.

Result: PolyJuice-steered T2I models deceive SIDs up to 84% more effectively than unsteered counterparts. Steering directions can be estimated at lower resolutions and transferred to higher ones via interpolation, reducing computational overhead.

Conclusion: PolyJuice enables effective black-box red-teaming of SIDs, and tuning SID models on PolyJuice-augmented datasets enhances detector performance by up to 30%.

Abstract: Synthetic image detectors (SIDs) are a key defense against the risks posed by the growing realism of images from text-to-image (T2I) models. Red teaming improves SID’s effectiveness by identifying and exploiting their failure modes via misclassified synthetic images. However, existing red-teaming solutions (i) require white-box access to SIDs, which is infeasible for proprietary state-of-the-art detectors, and (ii) generate image-specific attacks through expensive online optimization. To address these limitations, we propose PolyJuice, the first black-box, image-agnostic red-teaming method for SIDs, based on an observed distribution shift in the T2I latent space between samples correctly and incorrectly classified by the SID. PolyJuice generates attacks by (i) identifying the direction of this shift through a lightweight offline process that only requires black-box access to the SID, and (ii) exploiting this direction by universally steering all generated images towards the SID’s failure modes. PolyJuice-steered T2I models are significantly more effective at deceiving SIDs (up to 84%) compared to their unsteered counterparts. We also show that the steering directions can be estimated efficiently at lower resolutions and transferred to higher resolutions using simple interpolation, reducing computational overhead. Finally, tuning SID models on PolyJuice-augmented datasets notably enhances the performance of the detectors (up to 30%).

Method: Using grokking training regime where memorization precedes generalization, allowing temporal separation of phenomena. Testing models with enforced collapse, prevented collapse, and flatness regularization.

Result: Both neural collapse and relative flatness emerge near generalization onset, but only flatness consistently predicts generalization. Models regularized away from flat solutions exhibit delayed generalization resembling grokking.

Conclusion: Relative flatness is a potentially necessary and more fundamental property for generalization than neural collapse, which may cause flatness under classical assumptions but is not itself essential.

Abstract: Neural collapse, i.e., the emergence of highly symmetric, class-wise clustered representations, is frequently observed in deep networks and is often assumed to reflect or enable generalization. In parallel, flatness of the loss landscape has been theoretically and empirically linked to generalization. Yet, the causal role of either phenomenon remains unclear: Are they prerequisites for generalization, or merely by-products of training dynamics? We disentangle these questions using grokking, a training regime in which memorization precedes generalization, allowing us to temporally separate generalization from training dynamics and we find that while both neural collapse and relative flatness emerge near the onset of generalization, only flatness consistently predicts it. Models encouraged to collapse or prevented from collapsing generalize equally well, whereas models regularized away from flat solutions exhibit delayed generalization, resembling grokking, even in architectures and datasets where it does not typically occur. Furthermore, we show theoretically that neural collapse leads to relative flatness under classical assumptions, explaining their empirical co-occurrence. Our results support the view that relative flatness is a potentially necessary and more fundamental property for generalization, and demonstrate how grokking can serve as a powerful probe for isolating its geometric underpinnings.

Method: Developed two preference-driven selectors: GNN-preference-driven node selector for prediction distillation from LLMs to teacher GNNs, and node-preference-driven GNN selector to identify the most suitable teacher GNN for each node.

Result: Extensive experiments validate the framework’s efficacy in few-shot node classification on real-world TAGs.

Conclusion: The proposed PKD framework successfully combines complementary strengths of LLMs and GNNs for improved few-shot learning on text-attributed graphs.

Abstract: Graph neural networks (GNNs) can efficiently process text-attributed graphs (TAGs) due to their message-passing mechanisms, but their training heavily relies on the human-annotated labels. Moreover, the complex and diverse local topologies of nodes of real-world TAGs make it challenging for a single mechanism to handle. Large language models (LLMs) perform well in zero-/few-shot learning on TAGs but suffer from a scalability challenge. Therefore, we propose a preference-driven knowledge distillation (PKD) framework to synergize the complementary strengths of LLMs and various GNNs for few-shot node classification. Specifically, we develop a GNN-preference-driven node selector that effectively promotes prediction distillation from LLMs to teacher GNNs. To further tackle nodes’ intricate local topologies, we develop a node-preference-driven GNN selector that identifies the most suitable teacher GNN for each node, thereby facilitating tailored knowledge distillation from teacher GNNs to the student GNN. Extensive experiments validate the efficacy of our proposed framework in few-shot node classification on real-world TAGs. Our code is be available.

Method: Distributed learning method that integrates ML and wireless communication, leveraging all devices for distributed training of specialized compute efficient classifiers with minimal communication overhead, combined with tailored wireless multi-hop communication protocols.

Result: Hardware experiments on 20 ultra-low-power devices show RockNet learns timeseries classification from scratch, surpassing latest neural network microcontroller training accuracy by up to 2x, and reduces memory, latency and energy consumption by up to 90% when scaling from 1 to 20 devices.

Conclusion: Tight integration of distributed ML, distributed computing, and communication enables, for the first time, training on ultra-low-power hardware with state-of-the-art accuracy.

Abstract: As Machine Learning (ML) becomes integral to Cyber-Physical Systems (CPS), there is growing interest in shifting training from traditional cloud-based to on-device processing (TinyML), for example, due to privacy and latency concerns. However, CPS often comprise ultra-low-power microcontrollers, whose limited compute resources make training challenging. This paper presents RockNet, a new TinyML method tailored for ultra-low-power hardware that achieves state-of-the-art accuracy in timeseries classification, such as fault or malware detection, without requiring offline pretraining. By leveraging that CPS consist of multiple devices, we design a distributed learning method that integrates ML and wireless communication. RockNet leverages all devices for distributed training of specialized compute efficient classifiers that need minimal communication overhead for parallelization. Combined with tailored and efficient wireless multi-hop communication protocols, our approach overcomes the communication bottleneck that often occurs in distributed learning. Hardware experiments on a testbed with 20 ultra-low-power devices demonstrate RockNet’s effectiveness. It successfully learns timeseries classification tasks from scratch, surpassing the accuracy of the latest approach for neural network microcontroller training by up to 2x. RockNet’s distributed ML architecture reduces memory, latency and energy consumption per device by up to 90 % when scaling from one central device to 20 devices. Our results show that a tight integration of distributed ML, distributed computing, and communication enables, for the first time, training on ultra-low-power hardware with state-of-the-art accuracy.

Method: Leverages score-based diffusion models to estimate transfer entropy through conditional mutual information by learning score functions of relevant conditional distributions.

Result: Demonstrates superior accuracy and robustness compared to existing neural estimators and state-of-the-art approaches across synthetic benchmarks and real data.

Conclusion: TENDE provides flexible, scalable transfer entropy estimation while making minimal assumptions about the underlying data-generating process.

Abstract: Transfer entropy measures directed information flow in time series, and it has become a fundamental quantity in applications spanning neuroscience, finance, and complex systems analysis. However, existing estimation methods suffer from the curse of dimensionality, require restrictive distributional assumptions, or need exponentially large datasets for reliable convergence. We address these limitations in the literature by proposing TENDE (Transfer Entropy Neural Diffusion Estimation), a novel approach that leverages score-based diffusion models to estimate transfer entropy through conditional mutual information. By learning score functions of the relevant conditional distributions, TENDE provides flexible, scalable estimation while making minimal assumptions about the underlying data-generating process. We demonstrate superior accuracy and robustness compared to existing neural estimators and other state-of-the-art approaches across synthetic benchmarks and real data.

Method: Integrates strategic equilibrium modeling with doubly robust estimation, maintaining double robustness while incorporating strategic considerations under strategic unconfoundedness assumptions.

Result: Achieves 7.6%-29.3% bias reduction across varying strategic strengths compared to baseline methods, and demonstrates robust scalability with increasing agent populations.

Conclusion: SDR provides a principled and reliable framework for causal inference in strategic environments where agents respond strategically to interventions.

Abstract: We introduce the Strategic Doubly Robust (SDR) estimator, a novel framework that integrates strategic equilibrium modeling with doubly robust estimation for causal inference in strategic environments. SDR addresses endogenous treatment assignment arising from strategic agent behavior, maintaining double robustness while incorporating strategic considerations. Theoretical analysis confirms SDR’s consistency and asymptotic normality under strategic unconfoundedness. Empirical evaluations demonstrate SDR’s superior performance over baseline methods, achieving 7.6%-29.3% bias reduction across varying strategic strengths and maintaining robust scalability with agent populations. The framework provides a principled approach for reliable causal inference when agents respond strategically to interventions.

Method: Established a general lower bound of Ω(√T log(|X|)/log d) for any algorithm in combinatorial settings, analyzed specific combinatorial structures (m-sets, online multitask learning), and connected Hedge to Online Mirror Descent with dilated entropy regularizer for DAG shortest-path problems.

Result: Hedge is near-optimal (up to √log d factor) for general combinatorial sets, but provably suboptimal by exactly √log d for m-sets with log d ≤ m ≤ √d. Hedge is optimal for online multitask learning, and its near-optimality enables near-optimal regularizers for DAG shortest-path problems.

Conclusion: Hedge is generally near-optimal for combinatorial settings but not universally optimal - its performance depends on the specific combinatorial structure, with provable suboptimality in some cases and optimality in others.

Abstract: In this paper, we study the classical Hedge algorithm in combinatorial settings. In each round, the learner selects a vector $\boldsymbol{x}_t$ from a set $X \subseteq {0,1}^d$, observes a full loss vector $\boldsymbol{y}_t \in \mathbb{R}^d$, and incurs a loss $\langle \boldsymbol{x}_t, \boldsymbol{y}_t \rangle \in [-1,1]$. This setting captures several important problems, including extensive-form games, resource allocation, $m$-sets, online multitask learning, and shortest-path problems on directed acyclic graphs (DAGs). It is well known that Hedge achieves a regret of $O\big(\sqrt{T \log |X|}\big)$ after $T$ rounds of interaction. In this paper, we ask whether Hedge is optimal across all combinatorial settings. To that end, we show that for any $X \subseteq {0,1}^d$, Hedge is near-optimal–specifically, up to a $\sqrt{\log d}$ factor–by establishing a lower bound of $\Omega\big(\sqrt{T \log(|X|)/\log d}\big)$ that holds for any algorithm. We then identify a natural class of combinatorial sets–namely, $m$-sets with $\log d \leq m \leq \sqrt{d}$–for which this lower bound is tight, and for which Hedge is provably suboptimal by a factor of exactly $\sqrt{\log d}$. At the same time, we show that Hedge is optimal for online multitask learning, a generalization of the classical $K$-experts problem. Finally, we leverage the near-optimality of Hedge to establish the existence of a near-optimal regularizer for online shortest-path problems in DAGs–a setting that subsumes a broad range of combinatorial domains. Specifically, we show that the classical Online Mirror Descent (OMD) algorithm, when instantiated with the dilated entropy regularizer, is iterate-equivalent to Hedge, and therefore inherits its near-optimal regret guarantees for DAGs.

Method: The proposed method perturbs feature inputs directly instead of randomizing unknown parameters or adding noise to rewards. This approach is computationally efficient and naturally extends to non-parametric and neural network models.

Result: The algorithm achieves Õ(d√T) worst-case regret bound for generalized linear contextual bandits, improving upon the typical Õ(d³/²√T) regret of existing methods. Empirical evaluations show it surpasses existing methods while maintaining strong practical performance.

Conclusion: Feature perturbation provides a unified approach that combines near-optimal theoretical guarantees with practical computational efficiency, making it suitable for both parametric and non-parametric contextual bandit models.

Abstract: We propose feature perturbation, a simple yet effective exploration strategy for contextual bandits that injects randomness directly into feature inputs, instead of randomizing unknown parameters or adding noise to rewards. Remarkably, this algorithm achieves $\tilde{\mathcal{O}}(d\sqrt{T})$ worst-case regret bound for generalized linear contextual bandits, while avoiding the $\tilde{\mathcal{O}}(d^{3/2}\sqrt{T})$ regret typical of existing randomized bandit algorithms. Because our algorithm eschews parameter sampling, it is both computationally efficient and naturally extends to non-parametric or neural network models. We verify these advantages through empirical evaluations, demonstrating that feature perturbation not only surpasses existing methods but also unifies strong practical performance with the near-optimal regret guarantees.

Method: Uses conformal alignment-based cascading that treats edge-to-cloud escalation as multiple hypothesis testing, selecting which inputs can be safely handled at the edge while preserving statistical guarantees.

Result: Experiments on CIFAR-100 and TeleQnA show the method maintains target conditional coverage for edge predictions while substantially reducing cloud offloading with modest increases in prediction set size.

Conclusion: The CAb cascade provides statistical guarantees on edge decisions satisfying cloud-level conditional coverage, exposing a tunable trade-off among coverage, deferral rate, and set size.

Abstract: Edge intelligence enables low-latency inference via compact on-device models, but assuring reliability remains challenging. We study edge-cloud cascades that must preserve conditional coverage: whenever the edge returns a prediction set, it should contain the true label with a user-specified probability, as if produced by the cloud model. We formalize conditional coverage with respect to the cloud predictive distribution, and introduce a conformal alignment-based (CAb) cascading mechanism that certifies this property with user control over the risk level. Our method casts escalation from edge to cloud models as a multiple-hypothesis testing (MHT) problem, tailoring conformal alignment (CA) to select which inputs can be safely handled at the edge. The proposed CAb model cascading method yields statistical guarantees on the average fraction of edge decisions that satisfy cloud-level conditional coverage. The procedure applies to arbitrary edge prediction sets, including variants of conformal prediction (CP), and exposes a tunable trade-off among coverage, deferral rate, and set size. Experiments on CIFAR-100 image classification and the TeleQnA question-answering (QA) benchmark show that the proposed CAb cascade maintains the target conditional coverage for edge predictions while substantially reducing offloading to the cloud and incurring modest increases in prediction-set size.

Method: Extracts compact feature representations from images, performs SHAP-based feature attribution, uses RL-guided labeling for boundary datasets, and trains surrogate models (decision trees, regression functions) to interpret RL policy decisions.

Result: Maintains competitive task performance while substantially improving transparency and human understanding of agent behavior in Atari environments with three deep RL algorithms.

Conclusion: Transforms SILVER into a scalable, behavior-aware framework for interpreting deep RL agents in high-dimensional, multi-action settings, advancing explainable RL.

Abstract: Deep reinforcement learning (RL) achieves remarkable performance but lacks interpretability, limiting trust in policy behavior. The existing SILVER framework (Li, Siddique, and Cao 2025) explains RL policy via Shapley-based regression but remains restricted to low-dimensional, binary-action domains. We propose SILVER with RL-guided labeling, an enhanced variant that extends SILVER to multi-action and high-dimensional environments by incorporating the RL policy’s own action outputs into the boundary points identification. Our method first extracts compact feature representations from image observations, performs SHAP-based feature attribution, and then employs RL-guided labeling to generate behaviorally consistent boundary datasets. Surrogate models, such as decision trees and regression-based functions, are subsequently trained to interpret RL policy’s decision structure. We evaluate the proposed framework on two Atari environments using three deep RL algorithms and conduct human-subject study to assess the clarity and trustworthiness of the derived interpretable policy. Results show that our approach maintains competitive task performance while substantially improving transparency and human understanding of agent behavior. This work advances explainable RL by transforming SILVER into a scalable and behavior-aware framework for interpreting deep RL agents in high-dimensional, multi-action settings.

Method: Proposes Gaussian distillation using deep latent factor models to estimate teacher ensemble distributions. Uses EM algorithm to stably estimate mean and covariance functions, treating ensemble members as stochastic process realizations.

Result: Outperforms existing baselines on multiple benchmark datasets. Works well for language model fine-tuning and distribution shift problems.

Conclusion: Gaussian distillation effectively compresses ensembles while maintaining uncertainty quantification, enabling practical deployment to resource-constrained applications.

Abstract: Deep ensembles deliver state-of-the-art, reliable uncertainty quantification, but their heavy computational and memory requirements hinder their practical deployments to real applications such as on-device AI. Knowledge distillation compresses an ensemble into small student models, but existing techniques struggle to preserve uncertainty partly because reducing the size of DNNs typically results in variation reduction. To resolve this limitation, we introduce a new method of distribution distillation (i.e. compressing a teacher ensemble into a student distribution instead of a student ensemble) called Gaussian distillation, which estimates the distribution of a teacher ensemble through a special Gaussian process called the deep latent factor model (DLF) by treating each member of the teacher ensemble as a realization of a certain stochastic process. The mean and covariance functions in the DLF model are estimated stably by using the expectation-maximization (EM) algorithm. By using multiple benchmark datasets, we demonstrate that the proposed Gaussian distillation outperforms existing baselines. In addition, we illustrate that Gaussian distillation works well for fine-tuning of language models and distribution shift problems.

Method: Created CS-54k corpus from 14k CC-licensed papers using scalable paper-grounded pipeline with retrieval-augmented generation and multi-stage quality control. Derived CS-4k benchmark and CS-50k training dataset.

Result: CS-4k stratifies state-of-the-art LLMs into distinct capability tiers. Open models trained on CS-50k with supervised training and RL show substantial improvements, with 7B-scale models outperforming larger proprietary systems like GPT-4.1, GPT-4o, and Gemini 2.5 Pro.

Conclusion: Making AI models better research assistants relies more on domain-aligned training with high-quality data than on pretraining scale or general benchmark performance.

Abstract: As large language models (LLMs) advance, the ultimate vision for their role in science is emerging: we could build an AI collaborator to effectively assist human beings throughout the entire scientific research process. We refer to this envisioned system as ResearchGPT. Given that scientific research progresses through multiple interdependent phases, achieving this vision requires rigorous benchmarks that evaluate the end-to-end workflow rather than isolated sub-tasks. To this end, we contribute CS-54k, a high-quality corpus of scientific Q&A pairs in computer science, built from 14k CC-licensed papers. It is constructed through a scalable, paper-grounded pipeline that combines retrieval-augmented generation (RAG) with multi-stage quality control to ensure factual grounding. From this unified corpus, we derive two complementary subsets: CS-4k, a carefully curated benchmark for evaluating AI’s ability to assist scientific research, and CS-50k, a large-scale training dataset. Extensive experiments demonstrate that CS-4k stratifies state-of-the-art LLMs into distinct capability tiers. Open models trained on CS-50k with supervised training and reinforcement learning demonstrate substantial improvements. Even 7B-scale models, when properly trained, outperform many larger proprietary systems, such as GPT-4.1, GPT-4o, and Gemini 2.5 Pro. This indicates that making AI models better research assistants relies more on domain-aligned training with high-quality data than on pretraining scale or general benchmark performance. We release CS-4k and CS-50k in the hope of fostering AI systems as reliable collaborators in CS research.

Method: Develops a thresholding method that learns thresholds to tune mark probabilities normalized by prior probabilities, and a neural MTPP model that predicts mark first then time, avoiding expensive numerical integration.

Result: Extensive experiments on real-world datasets show superior performance for both mark and time prediction compared to various baselines.

Conclusion: The proposed solution effectively addresses the imbalanced mark distribution problem in MTPPs and improves prediction accuracy for rare events.

Abstract: Marked Temporal Point Process (MTPP) has been well studied to model the event distribution in marked event streams, which can be used to predict the mark and arrival time of the next event. However, existing studies overlook that the distribution of event marks is highly imbalanced in many real-world applications, with some marks being frequent but others rare. The imbalance poses a significant challenge to the performance of the next event prediction, especially for events of rare marks. To address this issue, we propose a thresholding method, which learns thresholds to tune the mark probability normalized by the mark’s prior probability to optimize mark prediction, rather than predicting the mark directly based on the mark probability as in existing studies. In conjunction with this method, we predict the mark first and then the time. In particular, we develop a novel neural MTPP model to support effective time sampling and estimation of mark probability without computationally expensive numerical improper integration. Extensive experiments on real-world datasets demonstrate the superior performance of our solution against various baselines for the next event mark and time prediction. The code is available at https://github.com/undes1red/IFNMTPP.

Method: A four-stage framework: (A) Structured Knowledge Curation, (B) Diversified Idea Generation, (C) Multi-stage Idea Selection, and (D) Expert Panel Review & Synthesis, with exposed reasoning states, execution logs, and tunable agents.

Result: Successfully demonstrated on a graph-mining case study (k-truss breaking problem), generating distinct, plausible hypotheses with evidence and critiques. The system is domain-agnostic and works with any scientific field that has literature sources.

Conclusion: AutoResearcher provides a transparent, controllable approach to automated literature-based ideation that produces diverse, evidence-aligned hypotheses while maintaining researcher oversight through its multi-agent framework.

Abstract: Effective research relies on organizing extensive information and stimulating novel solutions. Agentic systems have recently emerged as a promising tool to automate literature-based ideation. However, current systems often remain black-box. Their outputs may appear plausible but weakly grounded, with limited transparency or control for researchers. Our work introduces \textsc{autoresearcher}, a multi-agent demo system for knowledge-grounded and transparent ideation. Specifically, \textsc{autoresearcher} integrates meticulously designed four stages into a unified framework: (A) Structured Knowledge Curation, (B) Diversified Idea Generation, (C) Multi-stage Idea Selection, and (D) Expert Panel Review & Synthesis. Different from prior pipelines, our system not only exposes intermediate reasoning states, execution logs, and tunable agents for inspections, but also enables the generation of hypotheses that are both diverse and evidence-aligned. Our design is also domain-agnostic: as long as literature sources exist, the same pipeline can be instantiated in any scientific field. As an illustrative case, we demonstrate \textsc{autoresearcher} on a graph-mining case study ($k$-truss breaking problem), where it generates distinct, plausible hypotheses with evidence and critiques. A live demo and source code are available at https://github.com/valleysprings/AutoResearcher.

Method: Developed HIKMA framework with AI dataset curation, AI-based manuscript generation, AI-assisted peer review, AI-driven revision, AI conference presentation, and AI archival dissemination using language models and structured workflows with domain safeguards.

Result: Successfully implemented the HIKMA conference as a testbed and proof of concept, providing insights into opportunities and challenges of AI-enabled scholarship while maintaining intellectual property protection, transparency, and integrity.

Conclusion: The framework demonstrates AI can effectively support traditional scholarly practices through human-AI collaboration, while highlighting important questions about AI authorship, accountability, and the future role of AI in research.

Abstract: HIKMA Semi-Autonomous Conference is the first experiment in reimagining scholarly communication through an end-to-end integration of artificial intelligence into the academic publishing and presentation pipeline. This paper presents the design, implementation, and evaluation of the HIKMA framework, which includes AI dataset curation, AI-based manuscript generation, AI-assisted peer review, AI-driven revision, AI conference presentation, and AI archival dissemination. By combining language models, structured research workflows, and domain safeguards, HIKMA shows how AI can support - not replace traditional scholarly practices while maintaining intellectual property protection, transparency, and integrity. The conference functions as a testbed and proof of concept, providing insights into the opportunities and challenges of AI-enabled scholarship. It also examines questions about AI authorship, accountability, and the role of human-AI collaboration in research.

Method: ColorEcosystem consists of three key components: agent carrier (provides personalized service using user-specific data and digital twins), agent store (centralized platform for standardized agent management), and agent audit (ensures integrity through developer and user activity supervision).

Result: The proposed ColorEcosystem blueprint is designed to enable personalized, standardized, and trustworthy agentic service at scale across massive-agent ecosystems, with partial implementation already open-sourced.

Conclusion: ColorEcosystem provides a comprehensive solution to power personalized, standardized, and trustworthy agentic services in massive-agent ecosystems, addressing current limitations through its three-component architecture.

Abstract: With the rapid development of (multimodal) large language model-based agents, the landscape of agentic service management has evolved from single-agent systems to multi-agent systems, and now to massive-agent ecosystems. Current massive-agent ecosystems face growing challenges, including impersonal service experiences, a lack of standardization, and untrustworthy behavior. To address these issues, we propose ColorEcosystem, a novel blueprint designed to enable personalized, standardized, and trustworthy agentic service at scale. Concretely, ColorEcosystem consists of three key components: agent carrier, agent store, and agent audit. The agent carrier provides personalized service experiences by utilizing user-specific data and creating a digital twin, while the agent store serves as a centralized, standardized platform for managing diverse agentic services. The agent audit, based on the supervision of developer and user activities, ensures the integrity and credibility of both service providers and users. Through the analysis of challenges, transitional forms, and practical considerations, the ColorEcosystem is poised to power personalized, standardized, and trustworthy agentic service across massive-agent ecosystems. Meanwhile, we have also implemented part of ColorEcosystem’s functionality, and the relevant code is open-sourced at https://github.com/opas-lab/color-ecosystem.

Method: Proposes sequential agent modeling using diffusion models, treating multi-agent action revelation as reverse diffusion process to reduce complexity and capture agent dependencies.

Result: Achieves SOTA performance on MAMuJoCo and Bi-DexHands benchmarks, significantly improving final return and sample efficiency over prior world models.

Conclusion: DIMA establishes a new paradigm for multi-agent world models using diffusion models, advancing MARL research with improved modeling accuracy and efficiency.

Abstract: World models have recently attracted growing interest in Multi-Agent Reinforcement Learning (MARL) due to their ability to improve sample efficiency for policy learning. However, accurately modeling environments in MARL is challenging due to the exponentially large joint action space and highly uncertain dynamics inherent in multi-agent systems. To address this, we reduce modeling complexity by shifting from jointly modeling the entire state-action transition dynamics to focusing on the state space alone at each timestep through sequential agent modeling. Specifically, our approach enables the model to progressively resolve uncertainty while capturing the structured dependencies among agents, providing a more accurate representation of how agents influence the state. Interestingly, this sequential revelation of agents’ actions in a multi-agent system aligns with the reverse process in diffusion models–a class of powerful generative models known for their expressiveness and training stability compared to autoregressive or latent variable models. Leveraging this insight, we develop a flexible and robust world model for MARL using diffusion models. Our method, Diffusion-Inspired Multi-Agent world model (DIMA), achieves state-of-the-art performance across multiple multi-agent control benchmarks, significantly outperforming prior world models in terms of final return and sample efficiency, including MAMuJoCo and Bi-DexHands. DIMA establishes a new paradigm for constructing multi-agent world models, advancing the frontier of MARL research. Codes are open-sourced at https://github.com/breez3young/DIMA.

Method: Combined an agent-based model examining how semantic knowledge shapes cultural evolutionary dynamics with a large-scale behavioral experiment (N=1,243) testing semantic knowledge’s role in human innovation.

Result: Semantic knowledge directed exploration toward meaningful solutions and synergistically interacted with social learning to amplify innovation and cultural evolution. Participants without semantic knowledge performed no better than chance, even with social information, and used shallow exploration strategies.

Conclusion: Semantic knowledge is a key cognitive process enabling human cumulative culture.

Abstract: Cultural evolution allows ideas and technology to build over generations, a process reaching its most complex and open-ended form in humans. While social learning enables the transmission of such innovations, the cognitive processes that generate innovations remain unclear. We propose that semantic knowledge-the associations linking concepts to their properties and functions-guides human innovation and drives cumulative culture. To test this, we combined an agent-based model, which examines how semantic knowledge shapes cultural evolutionary dynamics, with a large-scale behavioural experiment (N = 1,243) testing its role in human innovation. Semantic knowledge directed exploration toward meaningful solutions and interacted synergistically with social learning to amplify innovation and cultural evolution. Participants lacking access to semantic knowledge performed no better than chance, even when social information was available, and relied on shallow exploration strategies for innovation. Together, these findings indicate that semantic knowledge is a key cognitive process enabling human cumulative culture.

Method: Uses step-wise reinforcement learning and self-evolving training for long-horizon interactions. Develops a tailored multi-agent framework for generality, consistency, and robustness. Explores personalized user intent recognition and proactive engagement.

Result: Achieved 77.2% success rate on AndroidWorld and 50.7% on AndroidLab benchmarks, establishing new state-of-the-art performance.

Conclusion: Current benchmarks are insufficient for comprehensive OS agent evaluation. Future work should focus on evaluation paradigms, agent collaboration, and security aspects.

Abstract: With the advancements in hardware, software, and large language model technologies, the interaction between humans and operating systems has evolved from the command-line interface to the rapidly emerging AI agent interactions. Building an operating system (OS) agent capable of executing user instructions and faithfully following user desires is becoming a reality. In this technical report, we present ColorAgent, an OS agent designed to engage in long-horizon, robust interactions with the environment while also enabling personalized and proactive user interaction. To enable long-horizon interactions with the environment, we enhance the model’s capabilities through step-wise reinforcement learning and self-evolving training, while also developing a tailored multi-agent framework that ensures generality, consistency, and robustness. In terms of user interaction, we explore personalized user intent recognition and proactive engagement, positioning the OS agent not merely as an automation tool but as a warm, collaborative partner. We evaluate ColorAgent on the AndroidWorld and AndroidLab benchmarks, achieving success rates of 77.2% and 50.7%, respectively, establishing a new state of the art. Nonetheless, we note that current benchmarks are insufficient for a comprehensive evaluation of OS agents and propose further exploring directions in future work, particularly in the areas of evaluation paradigms, agent collaboration, and security.

Method: Evaluated several state-of-the-art LALMs in two settings: interview (mixed speakers) and reading task (single child). Tasks included single-channel source separation to isolate child speech and child-only summarization preserving disfluencies. Used LLM as judge, human expert ratings, and BERTScore for evaluation.

Result: Findings delineate conditions under which LALMs produce faithful child-only summaries from mixed audio and identify where they fail. Reported agreement between models and between models and humans to assess reliability.

Conclusion: Provides practical guidance for clinical and educational deployments of LALMs for child speech analysis. Offers prompts and evaluation scripts to support replication.

Abstract: Child speech differs from adult speech in acoustics, prosody, and language development, and disfluencies (repetitions, prolongations, blocks) further challenge Automatic Speech Recognition (ASR) and downstream Natural Language Processing (NLP). Recent large audio-language models (LALMs) demonstrate strong cross-modal audio understanding; however, their behavior in disfluent child speech remains underexplored. We evaluate several state-of-the-art LALMs in two settings: an interview (mixed speakers) and a reading task (single child). The tasks are (i) single-channel source separation to isolate the child and (ii) child-only summarization that preserves clinically relevant disfluencies and avoids adult-speech leakage. Evaluation combines Large Language Model (LLM) as a judge, human expert ratings, and BERTScore (F1), and we report agreement between models and between models and humans to assess reliability. Our findings delineate the conditions under which LALMs produce faithful child-only summaries from mixed audio and where they fail, offering practical guidance for clinical and educational deployments. We provide prompts and evaluation scripts to support replication.

Method: Proposed Physiology-informed Multi-band Tokenization (PiMT) that decomposes ExG signals into 12 physiology-informed tokens followed by a reconstruction task to learn robust representations. Collected 50 hours of free-living ExG data using earphone-based hardware.

Result: PiMT consistently outperforms state-of-the-art methods across diverse tasks on the new DailySense dataset (first ExG dataset spanning five human senses) and four public ExG benchmarks.

Conclusion: The approach enables scalable, task-agnostic ExG monitoring in the wild by addressing data diversity gaps and providing adaptive feature recognition across the full frequency spectrum.

Abstract: Electrophysiological (ExG) signals offer valuable insights into human physiology, yet building foundation models that generalize across everyday tasks remains challenging due to two key limitations: (i) insufficient data diversity, as most ExG recordings are collected in controlled labs with bulky, expensive devices; and (ii) task-specific model designs that require tailored processing (i.e., targeted frequency filters) and architectures, which limit generalization across tasks. To address these challenges, we introduce an approach for scalable, task-agnostic ExG monitoring in the wild. We collected 50 hours of unobtrusive free-living ExG data with an earphone-based hardware prototype to narrow the data diversity gap. At the core of our approach is Physiology-informed Multi-band Tokenization (PiMT), which decomposes ExG signals into 12 physiology-informed tokens, followed by a reconstruction task to learn robust representations. This enables adaptive feature recognition across the full frequency spectrum while capturing task-relevant information. Experiments on our new DailySense dataset-the first to enable ExG-based analysis across five human senses-together with four public ExG benchmarks, demonstrate that PiMT consistently outperforms state-of-the-art methods across diverse tasks.

Method: The authors conducted controlled data-centric ablations focusing on three research questions: (1) processing raw web-crawled audio content for speech-text pretraining, (2) constructing synthetic pretraining datasets to augment web-crawled data, and (3) interleaving (text, audio) segments into training sequences.

Result: The insights from data-centric ablations were applied to pretrain a 3.8B-parameter SpeechLM called SpeLangy, which outperforms models up to 3x larger by 10.2% absolute performance.

Conclusion: Effective data curation has significant impact on speech-language pretraining, and the findings provide guidance for future data-centric exploration in SpeechLMs.

Abstract: Spoken Question-Answering (SQA) is a core capability for useful and interactive artificial intelligence systems. Recently, several speech-language models (SpeechLMs) have been released with a specific focus on improving their SQA performance. However, a lack of controlled ablations of pretraining data processing and curation makes it challenging to understand what factors account for performance, despite substantial gains from similar studies in other data modalities. In this work, we address this gap by conducting a data-centric exploration for pretraining SpeechLMs. We focus on three research questions fundamental to speech-language pretraining data: (1) how to process raw web-crawled audio content for speech-text pretraining, (2) how to construct synthetic pretraining datasets to augment web-crawled data and (3) how to interleave (text, audio) segments into training sequences. We apply the insights from our controlled data-centric ablations to pretrain a 3.8B-parameter SpeechLM, called SpeLangy, that outperforms models that are up to 3x larger by 10.2% absolute performance. We hope our findings highlight the impact of effective data curation for speech-language pretraining and guide future data-centric exploration in SpeechLMs.

Method: The proposed framework uses self-supervised learning (SSL) representations from both mixture and separated tracks to jointly predict audio quality (measured by SI-SNR) and speech intelligibility (measured by WER), eliminating the need for text references or ground truth audio.

Result: Experiments on WHAMR! dataset show WER estimation with MAE of 17% and PCC of 0.77; SI-SNR estimation with MAE of 1.38 and PCC of 0.95. The framework demonstrates robustness across various SSL representations.

Conclusion: The text-free reference-free evaluation framework using SSL representations provides an effective alternative to traditional evaluation methods, enabling assessment of speech separation systems in real-world scenarios where reference data is unavailable.

Abstract: Source separation is a crucial pre-processing step for various speech processing tasks, such as automatic speech recognition (ASR). Traditionally, the evaluation metrics for speech separation rely on the matched reference audios and corresponding transcriptions to assess audio quality and intelligibility. However, they cannot be used to evaluate real-world mixtures for which no reference exists. This paper introduces a text-free reference-free evaluation framework based on self-supervised learning (SSL) representations. The proposed framework utilize the mixture and separated tracks to predict jointly audio quality, through the Scale Invariant Signal to Noise Ratio (SI-SNR) metric, and speech intelligibility through the Word Error Rate (WER) metric. We conducted experiments on the WHAMR! dataset, which shows a WER estimation with a mean absolute error (MAE) of 17% and a Pearson correlation coefficient (PCC) of 0.77; and SI-SNR estimation with an MAE of 1.38 and PCC of 0.95. We further demonstrate the robustness of our estimator by using various SSL representations.

Method: Integrates asymmetric frequency-time architecture, Cyclical Calibration and Refinement (CCR) training strategy, and noise-invariant fine-tuning to optimize decoder resources and escape local optima.

Result: Ranked third overall in LRAC 2025 Challenge Track 1 and achieved best performance at 1 kbps in both noisy/reverberant conditions and intelligibility in clean tests.

Conclusion: PhoenixCodec effectively addresses the efficiency-quality trade-off in low-resource speech coding, demonstrating superior performance at extremely low bitrates.

Abstract: This paper presents PhoenixCodec, a comprehensive neural speech coding and decoding framework designed for extremely low-resource conditions. The proposed system integrates an optimized asymmetric frequency-time architecture, a Cyclical Calibration and Refinement (CCR) training strategy, and a noise-invariant fine-tuning procedure. Under stringent constraints - computation below 700 MFLOPs, latency less than 30 ms, and dual-rate support at 1 kbps and 6 kbps - existing methods face a trade-off between efficiency and quality. PhoenixCodec addresses these challenges by alleviating the resource scattering of conventional decoders, employing CCR to escape local optima, and enhancing robustness through noisy-sample fine-tuning. In the LRAC 2025 Challenge Track 1, the proposed system ranked third overall and demonstrated the best performance at 1 kbps in both real-world noise and reverberation and intelligibility in clean tests, confirming its effectiveness.

Method: Uses alternating CNN and RNN layers operating in the compressed spectral domain with multi-scale modeling to achieve greater efficiency and better representational capability.

Result: At 4 kbps, achieves comparable or superior performance to state-of-the-art lightweight codecs while using only 20% computation and 10% parameters, and significantly outperforms codecs with similar computational resources.

Conclusion: SpecTokenizer demonstrates that lightweight streaming neural audio codecs can achieve competitive performance through efficient spectral domain processing and multi-scale modeling.

Abstract: Neural Audio Codecs (NACs) have gained growing attention in recent years as technologies for audio compression and audio representation in speech language models. While mainstream NACs typically require G-level computation and M-level parameters, the performance of lightweight and streaming NACs remains underexplored. This paper proposes SpecTokenizer, a lightweight streaming codec that operates in the compressed spectral domain. Composed solely of alternating CNN and RNN layers, SpecTokenizer achieves greater efficiency and better representational capability through multi-scale modeling in the compressed spectrum domain. At 4 kbps, the proposed SpecTokenizer achieves comparable or superior performance compared to the codec with state-of-the-art lightweight architecture while requiring only 20% of the computation and 10% of the parameters. Furthermore, it significantly outperforms the codec when using similar computational and storage resources.

Method: The BPN extends existing lightweight SED systems by using intermediate latent representations from the backbone classification model to gate the final output, inspired by image object detection approaches. Two post-processing hyperparameter selection methods (forward-search and backward-search) are also introduced.

Result: BPN achieves 16.8% absolute increase in precision, with 21.3% and 9.4% F1-score improvements for minority-class d-calls and bp-calls respectively. The complete WhaleVAD-BPN system achieves 0.475 cross-validated F1-score, representing 9.8% absolute improvement over baseline.

Conclusion: The proposed BPN effectively addresses false positive and minority-class detection challenges in whale call detection, demonstrating that careful post-processing hyperparameter optimization combined with the boundary proposal approach leads to substantial performance gains in marine bioacoustics.

Abstract: While recent sound event detection (SED) systems can identify baleen whale calls in marine audio, challenges related to false positive and minority-class detection persist. We propose the boundary proposal network (BPN), which extends an existing lightweight SED system. The BPN is inspired by work in image object detection and aims to reduce the number of false positive detections. It achieves this by using intermediate latent representations computed within the backbone classification model to gate the final output. When added to an existing SED system, the BPN achieves a 16.8 % absolute increase in precision, as well as 21.3 % and 9.4 % improvements in the F1-score for minority-class d-calls and bp-calls, respectively. We further consider two approaches to the selection of post-processing hyperparameters: a forward-search and a backward-search. By separately optimising event-level and frame-level hyperparameters, these two approaches lead to considerable performance improvements over parameters selected using empirical methods. The complete WhaleVAD-BPN system achieves a cross-validated development F1-score of 0.475, which is a 9.8 % absolute improvement over the baseline.

Method: The authors propose a fully unsupervised approach using language models to factor in the detection of implausible sentences in synthesized speech, without needing reference signals.

Result: The paper presents a dataset of high-quality synthesized gibberish speech and provides code for calculating scores from various speech language models to assess implausible sentences in spoken language.

Conclusion: The work enables better detection of generative artifacts in speech synthesis through unsupervised language model-based methods and provides resources (dataset and code) for further development in this area.

Abstract: Significant research efforts are currently being dedicated to non-intrusive quality and intelligibility assessment, especially given how it enables curation of large scale datasets of in-the-wild speech data. However, with the increasing capabilities of generative models to synthesize high quality speech, new types of artifacts become relevant, such as generative hallucinations. While intrusive metrics are able to spot such sort of discrepancies from a reference signal, it is not clear how current non-intrusive methods react to high-quality phoneme confusions or, more extremely, gibberish speech. In this paper we explore how to factor in this aspect under a fully unsupervised setting by leveraging language models. Additionally, we publish a dataset of high-quality synthesized gibberish speech for further development of measures to assess implausible sentences in spoken language, alongside code for calculating scores from a variety of speech language models.

Method: The study explores knowledge distillation and pruning techniques to reduce QCNN complexity, with pruning proving more effective than knowledge distillation.

Result: Pruned QCNNs achieve competitive performance with conventional CNNs and Transformers while reducing computational cost by 50% and parameter count by 80% on AudioSet dataset, and generalize well across multiple audio classification benchmarks.

Conclusion: Pruning is an effective approach for reducing QCNN complexity while maintaining performance, making them more suitable for deployment on resource-constrained platforms.

Abstract: Conventional Convolutional Neural Networks (CNNs) in the real domain have been widely used for audio classification. However, their convolution operations process multi-channel inputs independently, limiting the ability to capture correlations among channels. This can lead to suboptimal feature learning, particularly for complex audio patterns such as multi-channel spectrogram representations. Quaternion Convolutional Neural Networks (QCNNs) address this limitation by employing quaternion algebra to jointly capture inter-channel dependencies, enabling more compact models with fewer learnable parameters while better exploiting the multi-dimensional nature of audio signals. However, QCNNs exhibit higher computational complexity due to the overhead of quaternion operations, resulting in increased inference latency and reduced efficiency compared to conventional CNNs, posing challenges for deployment on resource-constrained platforms. To address this challenge, this study explores knowledge distillation (KD) and pruning, to reduce the computational complexity of QCNNs while maintaining performance. Our experiments on audio classification reveal that pruning QCNNs achieves similar or superior performance compared to KD while requiring less computational effort. Compared to conventional CNNs and Transformer-based architectures, pruned QCNNs achieve competitive performance with a reduced learnable parameter count and computational complexity. On the AudioSet dataset, pruned QCNNs reduce computational cost by 50% and parameter count by 80%, while maintaining performance comparable to the conventional CNNs. Furthermore, pruned QCNNs generalize well across multiple audio classification benchmarks, including GTZAN for music genre recognition, ESC-50 for environmental sound classification and RAVDESS for speech emotion recognition.

Method: Uses MP-ADM (magnitude-preserving ablated diffusion model) architecture as denoiser, incorporates visual features with MP-FiLM (magnitude-preserving feature-wise linear modulation), adds speaker embeddings, and reconstructs speech waveform with neural vocoder from generated mel-spectrograms.

Result: Outperforms existing lip-to-speech baselines on LRS3 dataset in perceptual speech quality and speaker similarity, remains competitive in downstream automatic speech recognition, with findings supported by formal listening experiment.

Conclusion: LipDiffuser demonstrates superior performance in lip-to-speech generation, producing high-quality, intelligible speech that closely matches the speaker’s characteristics while maintaining competitive ASR performance.

Abstract: We present LipDiffuser, a conditional diffusion model for lip-to-speech generation synthesizing natural and intelligible speech directly from silent video recordings. Our approach leverages the magnitude-preserving ablated diffusion model (MP-ADM) architecture as a denoiser model. To effectively condition the model, we incorporate visual features using magnitude-preserving feature-wise linear modulation (MP-FiLM) alongside speaker embeddings. A neural vocoder then reconstructs the speech waveform from the generated mel-spectrograms. Evaluations on LRS3 demonstrate that LipDiffuser outperforms existing lip-to-speech baselines in perceptual speech quality and speaker similarity, while remaining competitive in downstream automatic speech recognition. These findings are also supported by a formal listening experiment.

Method: Dual-head architecture with diffusion head for continuous speech representations (frame-level autoregressive) and original LM head for multitask capability; masked training for exposure bias; two-stage training scheme with frozen LM in second stage.

Result: Achieved WER of 1.95%, speaker similarity of 0.54, and UTMOS of 4.00 on LibriSpeech test-clean; two-stage training yields 46% relative WER reduction over one-stage baseline.

Conclusion: Combining autoregressive modeling with continuous-token diffusion through two-stage training is effective for high-quality TTS in MLLM frameworks.

Abstract: Unified architectures in multimodal large language models (MLLM) have shown promise in handling diverse tasks within a single framework. In the text-to-speech (TTS) task, current MLLM-based approaches rely on discrete token representations, which disregard the inherently continuous nature of speech and can lead to loss of fine-grained acoustic information. In this work, we investigate the TTS within the MLLM paradigm using continuous speech representations. We design a dual-head architecture and implement two complementary training strategies for a robust model. (1) A diffusion head generating continuous speech representations is added on the MLLM, which is on frame-level and strictly autoregressive. (2) The original language model head is retained to preserve multitask capability and to control the start and end of speech synthesis. (3) Masked training is employed to address exposure bias in autoregressive decoding. (4) To stabilize optimization, we propose a two-stage scheme where the LM is frozen in the second stage, ensuring the diffusion head learns from a fixed input distribution. Evaluations on LibriSpeech(PC) test-clean show that our approach achieves state-of-the-art autoregressive performance, with a WER of 1.95%, speaker similarity of 0.54, and UTMOS of 4.00. The two-stage training yields a 46% relative WER reduction over the one-stage training baseline. These results highlight the effectiveness of combining autoregressive modeling with continuous-token diffusion, supported by a two-stage training procedure.

Method: Analyze ultrasound TPI signals with preprocessing (z-score normalization, PCA dimensionality reduction), evaluate multiple classifiers across three feature representations: original 31D space, reduced subset, and PCA-transformed space.

Result: PCA transformation substantially improves performance, with ensemble methods achieving 98.0% accuracy and F1-score of 0.890, effectively handling class imbalance.

Conclusion: Machine learning-based ICH detection using portable ultrasound is feasible and applicable in emergency medicine, rural healthcare, and military settings where traditional imaging is unavailable.

Abstract: Intracranial hemorrhage (ICH) secondary to Traumatic Brain Injury (TBI) represents a critical diagnostic challenge, with approximately 64,000 TBI-related deaths annually in the United States. Current diagnostic modalities including Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) have significant limitations: high cost, limited availability, and infrastructure dependence, particularly in resource-constrained environments. This study investigates machine learning approaches for automated ICH detection using Ultrasound Tissue Pulsatility Imaging (TPI), a portable technique measuring tissue displacement from hemodynamic forces during cardiac cycles. We analyze ultrasound TPI signals comprising 30 temporal frames per cardiac cycle with recording angle information, collected from TBI patients with CT-confirmed ground truth labels. Our preprocessing pipeline employs z-score normalization and Principal Component Analysis (PCA) for dimensionality reduction, retaining components explaining 95% of cumulative variance. We systematically evaluate multiple classification algorithms spanning probabilistic, kernel-based, neural network, and ensemble learning approaches across three feature representations: original 31-dimensional space, reduced subset, and PCA-transformed space. Results demonstrate that PCA transformation substantially improves classifier performance, with ensemble methods achieving 98.0% accuracy and F1-score of 0.890, effectively balancing precision and recall despite class imbalance. These findings establish the feasibility of machine learning-based ICH detection in TBI patients using portable ultrasound devices, with applications in emergency medicine, rural healthcare, and military settings where traditional imaging is unavailable.

Method: Pilot study with 3 radiologists interpreting 180 chest radiographs twice (with/without bounding boxes) using eye tracking. Metrics included interpretation time, time to first lesion fixation, lesion dwell time, gaze-path length, and lung coverage. Linear mixed model analysis on true positives.

Result: Bounding-box displays prolonged interpretation time by 4.9s, increased lesion dwell time by 1.3s, increased gaze-path length by 2076 pixels, increased lung coverage by 10.5%, and reduced time to first lesion fixation by 1.3s (all p<0.001).

Conclusion: Eye tracking successfully captured measurable alterations in search behavior from concurrent bounding-box displays, supporting feasibility of this approach and need for larger studies to confirm effects across clinical contexts.

Abstract: Rationale and Objectives: Computer-aided detection systems for chest radiographs are widely used, and concurrent reader displays, such as bounding-box (BB) highlights, may influence the reading process. This pilot study used eye tracking to conduct a preliminary experiment to quantify which aspects of visual search were affected. Materials and Methods: We sampled 180 chest radiographs from the VinDR-CXR dataset: 120 with solitary pulmonary nodules or masses and 60 without. The BBs were configured to yield an overall display sensitivity and specificity of 80%. Three radiologists (with 11, 5, and 1 years of experience, respectively) interpreted each case twice - once with BBs visible and once without - after a washout of >= 2 weeks. Eye movements were recorded using an EyeTech VT3 Mini. Metrics included interpretation time, time to first fixation on the lesion, lesion dwell time, total gaze-path length, and lung-field coverage ratio. Outcomes were modeled using a linear mixed model, with reading condition as a fixed effect and case and reader as random intercepts. The primary analysis was restricted to true positives (n=96). Results: Concurrent BB display prolonged interpretation time by 4.9 s (p<0.001) and increased lesion dwell time by 1.3 s (p<0.001). Total gaze-path length increased by 2,076 pixels (p<0.001), and lung-field coverage ratio increased by 10.5% (p<0.001). Time to first fixation on the lesion was reduced by 1.3 s (p<0.001). Conclusion: Eye tracking captured measurable alterations in search behavior associated with concurrent BB displays during chest radiograph interpretation. These findings support the feasibility of this approach and highlight the need for larger studies to confirm effects and explore implications across modalities and clinical contexts.

Method: Ensemble approach combining baseline SegResNet, attention-augmented SegResNet with concatenative skip connections, and dual-decoder U-Net with attention-gated skip connections, trained for only 20 epochs.

Result: Achieved competitive performance on BraTS-MEN 2025 dataset with Lesion-Wise Dice scores of 77.30% (ET), 76.37% (TC), and 73.9% (WT) on test data.

Conclusion: The ensemble method provides an effective and accessible tool for meningioma segmentation, demonstrating the value of architectural diversity even under hardware constraints.

Abstract: Meningiomas represent the most prevalent form of primary brain tumors, comprising nearly one-third of all diagnosed cases. Accurate delineation of these tumors from MRI scans is crucial for guiding treatment strategies, yet remains a challenging and time-consuming task in clinical practice. Recent developments in deep learning have accelerated progress in automated tumor segmentation; however, many advanced techniques are hindered by heavy computational demands and long training schedules, making them less accessible for researchers and clinicians working with limited hardware. In this work, we propose a novel ensemble-based segmentation approach that combines three distinct architectures: (1) a baseline SegResNet model, (2) an attention-augmented SegResNet with concatenative skip connections, and (3) a dual-decoder U-Net enhanced with attention-gated skip connections (DDUNet). The ensemble aims to leverage architectural diversity to improve robustness and accuracy while significantly reducing training demands. Each baseline model was trained for only 20 epochs and Evaluated on the BraTS-MEN 2025 dataset. The proposed ensemble model achieved competitive performance, with average Lesion-Wise Dice scores of 77.30%, 76.37% and 73.9% on test dataset for Enhancing Tumor (ET), Tumor Core (TC) and Whole Tumor (WT) respectively. These results highlight the effectiveness of ensemble learning for brain tumor segmentation, even under limited hardware constraints. Our proposed method provides a practical and accessible tool for aiding the diagnosis of meningioma, with potential impact in both clinical and research settings.

Method: Developed Adaptive Focal Loss (A-FL) that dynamically adjusts based on object surface smoothness, size, and class balancing parameter (target area to background ratio).

Result: A-FL achieved IoU of 0.696 and DSC of 0.769 on PICAI 2022 (5.5% and 5.4% improvement over Focal Loss), and IoU of 0.883 and DSC of 0.931 on BraTS 2018, outperforming all baseline losses by significant margins.

Conclusion: The proposed Adaptive Focal Loss effectively addresses challenges in medical image segmentation for complex anatomical regions and demonstrates superior performance compared to conventional loss functions.

Abstract: Deep learning has achieved remarkable accuracy in medical image segmentation, particularly for larger structures with well-defined boundaries. However, its effectiveness can be challenged by factors such as irregular object shapes and edges, non-smooth surfaces, small target areas, etc. which complicate the ability of networks to grasp the intricate and diverse nature of anatomical regions. In response to these challenges, we propose an Adaptive Focal Loss (A-FL) that takes both object boundary smoothness and size into account, with the goal to improve segmentation performance in intricate anatomical regions. The proposed A-FL dynamically adjusts itself based on an object’s surface smoothness, size, and the class balancing parameter based on the ratio of targeted area and background. We evaluated the performance of the A-FL on the PICAI 2022 and BraTS 2018 datasets. In the PICAI 2022 dataset, the A-FL achieved an Intersection over Union (IoU) score of 0.696 and a Dice Similarity Coefficient (DSC) of 0.769, outperforming the regular Focal Loss (FL) by 5.5% and 5.4% respectively. It also surpassed the best baseline by 2.0% and 1.2%. In the BraTS 2018 dataset, A-FL achieved an IoU score of 0.883 and a DSC score of 0.931. Our ablation experiments also show that the proposed A-FL surpasses conventional losses (this includes Dice Loss, Focal Loss, and their hybrid variants) by large margin in IoU, DSC, and other metrics. The code is available at https://github.com/rakibuliuict/AFL-CIBM.git.

Method: AIDFusion uses a disentangle Transformer to filter inconsistent atlas-specific information and distill distinguishable connections across atlases. It incorporates subject- and population-level consistency constraints, and employs an inter-atlas message-passing mechanism to fuse complementary information across brain regions.

Result: Experimental results on four disease datasets demonstrate AIDFusion’s effectiveness and efficiency compared to state-of-the-art methods. A case study shows that AIDFusion extracts interpretable patterns consistent with established neuroscience findings.

Conclusion: AIDFusion successfully addresses the challenges of multi-atlas brain network classification by ensuring cross-atlas consistency and enabling effective information fusion, providing improved performance and interpretable results for neurological disorder analysis.

Abstract: In the realm of neuroscience, identifying distinctive patterns associated with neurological disorders via brain networks is crucial. Resting-state functional magnetic resonance imaging (fMRI) serves as a primary tool for mapping these networks by correlating blood-oxygen-level-dependent (BOLD) signals across different brain regions, defined as regions of interest (ROIs). Constructing these brain networks involves using atlases to parcellate the brain into ROIs based on various hypotheses of brain division. However, there is no standard atlas for brain network classification, leading to limitations in detecting abnormalities in disorders. Some recent methods have proposed utilizing multiple atlases, but they neglect consistency across atlases and lack ROI-level information exchange. To tackle these limitations, we propose an Atlas-Integrated Distillation and Fusion network (AIDFusion) to improve brain network classification using fMRI data. AIDFusion addresses the challenge of utilizing multiple atlases by employing a disentangle Transformer to filter out inconsistent atlas-specific information and distill distinguishable connections across atlases. It also incorporates subject- and population-level consistency constraints to enhance cross-atlas consistency. Additionally, AIDFusion employs an inter-atlas message-passing mechanism to fuse complementary information across brain regions. Experimental results on four datasets of different diseases demonstrate the effectiveness and efficiency of AIDFusion compared to state-of-the-art methods. A case study illustrates AIDFusion extract patterns that are both interpretable and consistent with established neuroscience findings.

Method: Proposes I²SB-Inversion framework based on Schrödinger Bridge that performs pixel-wise translation between paired contrasts and introduces an inversion strategy to correct inter-modality misalignment.

Result: Achieves acceleration factor of up to 14.4 and consistently outperforms existing methods in both quantitative and qualitative evaluations on paired T1- and T2-weighted datasets.

Conclusion: The proposed method effectively utilizes cross-contrast priors through explicit structural constraints and misalignment correction, demonstrating superior performance in multi-contrast MRI reconstruction.

Abstract: Magnetic Resonance Imaging (MRI) is an inherently multi-contrast modality, where cross-contrast priors can be exploited to improve image reconstruction from undersampled data. Recently, diffusion models have shown remarkable performance in MRI reconstruction. However, they still struggle to effectively utilize such priors, mainly because existing methods rely on feature-level fusion in image or latent spaces, which lacks explicit structural correspondence and thus leads to suboptimal performance. To address this issue, we propose $\mathbf{I}^2$SB-Inversion, a multi-contrast guided reconstruction framework based on the Schr"odinger Bridge (SB). The proposed method performs pixel-wise translation between paired contrasts, providing explicit structural constraints between the guidance and target images. Furthermore, an Inversion strategy is introduced to correct inter-modality misalignment, which often occurs in guided reconstruction, thereby mitigating artifacts and improving reconstruction accuracy. Experiments on paired T1- and T2-weighted datasets demonstrate that $\mathbf{I}^2$SB-Inversion achieves a high acceleration factor of up to 14.4 and consistently outperforms existing methods in both quantitative and qualitative evaluations.

Method: Investigated diverse INRs across 2D/3D real and synthetic signals with varying bandwidth, testing overfitting and generalization tasks including tomography, super-resolution, and denoising. Stratified performance by model size, signal type, and bandwidth.

Result: Regularized grid with interpolation trains faster and achieves higher quality than any INR with same parameter count for most tasks and signals. INRs only outperform grids in limited settings like fitting binary signals (shape contours).

Conclusion: Future INR development should focus on applications where they show advantages, particularly binary signal fitting tasks, rather than general-purpose use where grids perform better.

Abstract: Implicit Neural Representations (INRs) have recently shown impressive results, but their fundamental capacity, implicit biases, and scaling behavior remain poorly understood. We investigate the performance of diverse INRs across a suite of 2D and 3D real and synthetic signals with varying effective bandwidth, as well as both overfitting and generalization tasks including tomography, super-resolution, and denoising. By stratifying performance according to model size as well as signal type and bandwidth, our results shed light on how different INR and grid representations allocate their capacity. We find that, for most tasks and signals, a simple regularized grid with interpolation trains faster and to higher quality than any INR with the same number of parameters. We also find limited settings–namely fitting binary signals such as shape contours–where INRs outperform grids, to guide future development and use of INRs towards the most advantageous applications.

Method: During video encoding, monitors cache and runs lightweight compression when threshold is reached: (1) removes temporally redundant tokens via Temporal-axis Redundancy (TaR) metric, (2) keeps semantically significant tokens via Value-Norm (VaN) ranking.

Result: Cuts peak GPU memory by up to 94%, sustains real-time generation, and matches or surpasses full-cache accuracy across four MLLMs and four benchmarks, including multi-turn dialogues.

Conclusion: Dissolves the KV cache bottleneck without retraining or query knowledge, enabling on-device streaming video assistants.

Abstract: Modern multimodal large language models (MLLMs) can reason over hour-long video, yet their key-value (KV) cache grows linearly with time-quickly exceeding the fixed memory of phones, AR glasses, and edge robots. Prior compression schemes either assume the whole video and user query are available offline or must first build the full cache, so memory still scales with stream length. InfiniPot-V is the first training-free, query-agnostic framework that enforces a hard, length-independent memory cap for streaming video understanding. During video encoding it monitors the cache and, once a user-set threshold is reached, runs a lightweight compression pass that (i) removes temporally redundant tokens via Temporal-axis Redundancy (TaR) metric and (ii) keeps semantically significant tokens via Value-Norm (VaN) ranking. Across four open-source MLLMs and four long-video and streaming-video benchmarks, InfiniPot-V cuts peak GPU memory by up to 94%, sustains real-time generation, and matches or surpasses full-cache accuracy-even in multi-turn dialogues. By dissolving the KV cache bottleneck without retraining or query knowledge, InfiniPot-V closes the gap for on-device streaming video assistants.

Method: A fully convolutional deep learning approach with spatial feature extraction from volumetric PET time frames, followed by temporal processing to predict arterial input functions.

Result: The model reliably predicts input functions with low error and high correlation, works on truncated/shifted scans, but fails on different radiotracers not in training data.

Conclusion: FC-DLIF provides a non-invasive, reliable alternative to blood sampling that is robust to temporal variations and scan duration changes.

Abstract: Dynamic positron emission tomography (PET) and kinetic modeling are pivotal in advancing tracer development research in small animal studies. Accurate kinetic modeling requires precise input function estimation, traditionally achieved via arterial blood sampling. However, arterial cannulation in small animals like mice, involves intricate, time-consuming, and terminal procedures, precluding longitudinal studies. This work proposes a non-invasive, fully convolutional deep learning-based approach (FC-DLIF) to predict input functions directly from PET imaging, potentially eliminating the need for blood sampling in dynamic small-animal PET. The proposed FC-DLIF model includes a spatial feature extractor acting on the volumetric time frames of the PET sequence, extracting spatial features. These are subsequently further processed in a temporal feature extractor that predicts the arterial input function. The proposed approach is trained and evaluated using images and arterial blood curves from [$^{18}$F]FDG data using cross validation. Further, the model applicability is evaluated on imaging data and arterial blood curves collected using two additional radiotracers ([$^{18}$F]FDOPA, and [$^{68}$Ga]PSMA). The model was further evaluated on data truncated and shifted in time, to simulate shorter, and shifted, PET scans. The proposed FC-DLIF model reliably predicts the arterial input function with respect to mean squared error and correlation. Furthermore, the FC-DLIF model is able to predict the arterial input function even from truncated and shifted samples. The model fails to predict the AIF from samples collected using different radiotracers, as these are not represented in the training data. Our deep learning-based input function offers a non-invasive and reliable alternative to arterial blood sampling, proving robust and flexible to temporal shifts and different scan durations.

Method: Proposes Robust Residual Finite Scalar Quantization (RFSQ) with two novel conditioning strategies: learnable scaling factors and invertible layer normalization to maintain normalized input statistics across stages.

Result: RFSQ-LayerNorm achieves 3.646 DNSMOS (3.6% improvement) in audio reconstruction and 0.102 L1 loss/0.100 perceptual loss on ImageNet (9.7% L1 improvement, 17.4% perceptual improvement over unconditioned variants).

Conclusion: RFSQ combines FSQ’s simplicity with multi-stage quantization’s representational power, establishing a new standard for neural compression across diverse modalities.

Abstract: Finite Scalar Quantization (FSQ) offers simplified training but suffers from residual magnitude decay in multi-stage settings, where subsequent stages receive exponentially weaker signals. We propose Robust Residual Finite Scalar Quantization (RFSQ), addressing this fundamental limitation through two novel conditioning strategies: learnable scaling factors and invertible layer normalization. Our experiments across audio and image modalities demonstrate RFSQ’s effectiveness and generalizability. In audio reconstruction at 24 bits/frame, RFSQ-LayerNorm achieves 3.646 DNSMOS, a 3.6% improvement over state-of-the-art RVQ (3.518). On ImageNet, RFSQ achieves 0.102 L1 loss and 0.100 perceptual loss, with LayerNorm providing 9.7% L1 improvement and 17.4% perceptual improvement over unconditioned variants. The LayerNorm strategy consistently outperforms alternatives by maintaining normalized input statistics across stages, effectively preventing exponential magnitude decay that limits naive residual approaches. RFSQ combines FSQ’s simplicity with multi-stage quantization’s representational power, establishing a new standard for neural compression across diverse modalities.

Method: Two-level hierarchical soft mixture-of-experts (HoME) built on Mamba SSM backbone: first level partitions sequences into local groups with per-group experts, second level aggregates outputs through global SMoE for cross-group fusion and global context refinement.

Result: Surpasses state-of-the-art results across datasets from three most widely used 3D medical imaging modalities and varying data qualities.

Conclusion: Hierarchical design combining local expert routing with global expert refinement enhances generalizability and segmentation performance for 3D medical images.

Abstract: In recent years, artificial intelligence has significantly advanced medical image segmentation. Nonetheless, challenges remain, including efficient 3D medical image processing across diverse modalities and handling data variability. In this work, we introduce Hierarchical Soft Mixture-of-Experts (HoME), a two-level token-routing layer for efficient long-context modeling, specifically designed for 3D medical image segmentation. Built on the Mamba Selective State Space Model (SSM) backbone, HoME enhances sequential modeling through adaptive expert routing. In the first level, a Soft Mixture-of-Experts (SMoE) layer partitions input sequences into local groups, routing tokens to specialized per-group experts for localized feature extraction. The second level aggregates these outputs through a global SMoE layer, enabling cross-group information fusion and global context refinement. This hierarchical design, combining local expert routing with global expert refinement, enhances generalizability and segmentation performance, surpassing state-of-the-art results across datasets from the three most widely used 3D medical imaging modalities and varying data qualities. The code is publicly available at https://github.com/gmum/MambaHoME.

Method: Generates 3D assets from single images with accurate geometry, well-aligned textures, and realistic physically-based materials that can be directly integrated into physics engines.

Result: Produces simulation-ready assets that enable deployment in robotic manipulation and simulation training, and scales to complete scene generation through object assembly.

Conclusion: Seed3D 1.0 provides a foundation for advancing physics-based world simulators by enabling scalable simulation-ready content creation.

Abstract: Developing embodied AI agents requires scalable training environments that balance content diversity with physics accuracy. World simulators provide such environments but face distinct limitations: video-based methods generate diverse content but lack real-time physics feedback for interactive learning, while physics-based engines provide accurate dynamics but face scalability limitations from costly manual asset creation. We present Seed3D 1.0, a foundation model that generates simulation-ready 3D assets from single images, addressing the scalability challenge while maintaining physics rigor. Unlike existing 3D generation models, our system produces assets with accurate geometry, well-aligned textures, and realistic physically-based materials. These assets can be directly integrated into physics engines with minimal configuration, enabling deployment in robotic manipulation and simulation training. Beyond individual objects, the system scales to complete scene generation through assembling objects into coherent environments. By enabling scalable simulation-ready content creation, Seed3D 1.0 provides a foundation for advancing physics-based world simulators. Seed3D 1.0 is now available on https://console.volcengine.com/ark/region:ark+cn-beijing/experience/vision?modelId=doubao-seed3d-1-0-250928&tab=Gen3D