Method: Uses sparse autoencoders (SAEs) to extract concept activations, computes saliency-weighted performance scores across benchmark data to ground evaluation in model’s internal representations.

Result: Found models consistently underperform on concepts related to non-sycophantic behaviors and safety discussions; benchmarks over-represent obedience/authority concepts while missing core intended concepts.

Conclusion: The method provides concept-level decomposition of benchmark scores, revealing why models scored as they did and how benchmarks could better reflect their intended scope, complementing conventional aggregated metrics.

Abstract: The evaluation of large language models (LLMs) relies heavily on standardized benchmarks. These benchmarks provide useful aggregated metrics for a given capability, but those aggregated metrics can obscure (i) particular sub-areas where the LLMs are weak (“model gaps”) and (ii) imbalanced coverage in the benchmarks themselves (“benchmark gaps”). We propose a new method that uses sparse autoencoders (SAEs) to automatically uncover both types of gaps. By extracting SAE concept activations and computing saliency-weighted performance scores across benchmark data, the method grounds evaluation in the model’s internal representations and enables comparison across benchmarks. As examples demonstrating our approach, we applied the method to two popular open-source models and ten benchmarks. We found that these models consistently underperformed on concepts that stand in contrast to sycophantic behaviors (e.g., politely refusing a request or asserting boundaries) and concepts connected to safety discussions. These model gaps align with observations previously surfaced in the literature; our automated, unsupervised method was able to recover them without manual supervision. We also observed benchmark gaps: many of the evaluated benchmarks over-represented concepts related to obedience, authority, or instruction-following, while missing core concepts that should fall within their intended scope. In sum, our method offers a representation-grounded approach to evaluation, enabling concept-level decomposition of benchmark scores. Rather than replacing conventional aggregated metrics, CG complements them by providing a concept-level decomposition that can reveal why a model scored as it did and how benchmarks could evolve to better reflect their intended scope. Code is available at https://competency-gaps.github.io.

Method: SA-DiffuSeq integrates sparse attention into the diffusion framework, using selective attention allocation to reduce computational complexity. A key innovation is a soft absorbing state tailored to sparse attention dynamics that stabilizes diffusion trajectories and accelerates sequence reconstruction.

Result: SA-DiffuSeq consistently surpasses state-of-the-art diffusion baselines in both training efficiency and sampling speed, with especially strong gains on extended sequences. It maintains semantic coherence and generation quality while significantly reducing computational requirements.

Conclusion: Incorporating structured sparsity into diffusion models is a promising direction for efficient and expressive long text generation, making SA-DiffuSeq well-suited for demanding applications like scientific writing, large-scale code generation, and multi-turn long-context dialogue.

Abstract: Diffusion based approaches to long form text generation suffer from prohibitive computational cost and memory overhead as sequence length increases. We introduce SA-DiffuSeq, a diffusion framework that integrates sparse attention to fundamentally improve scalability for long document modeling. By selectively allocating attention within the diffusion process, SA-DiffuSeq significantly reduces computational complexity while maintaining semantic coherence and generation quality. A key component of our method is a soft absorbing state tailored to sparse attention dynamics, which stabilizes diffusion trajectories and accelerates sequence reconstruction. This design improves sampling efficiency and enhances precision in long range dependency modeling. Extensive experiments demonstrate that SA-DiffuSeq consistently surpasses state of the art diffusion baselines in both training efficiency and sampling speed, with especially strong gains on extended sequences. These properties make SA-DiffuSeq well suited for demanding long form applications such as scientific writing, large scale code generation, and multi turn long context dialogue. Overall, our results indicate that incorporating structured sparsity into diffusion models is a promising direction for efficient and expressive long text generation.

Method: Created TokSuite with: 1) 14 models using different tokenizers but identical architecture, dataset, training budget, and initialization; 2) A new benchmark measuring model performance on real-world perturbations that affect tokenization.

Result: TokSuite enables robust decoupling of tokenizer influence, supporting novel findings about benefits and shortcomings of various popular tokenizers.

Conclusion: TokSuite provides a systematic framework for studying tokenization’s role in language models, revealing insights about different tokenizers’ impacts on LM performance.

Abstract: Tokenizers provide the fundamental basis through which text is represented and processed by language models (LMs). Despite the importance of tokenization, its role in LM performance and behavior is poorly understood due to the challenge of measuring the impact of tokenization in isolation. To address this need, we present TokSuite, a collection of models and a benchmark that supports research into tokenization’s influence on LMs. Specifically, we train fourteen models that use different tokenizers but are otherwise identical using the same architecture, dataset, training budget, and initialization. Additionally, we curate and release a new benchmark that specifically measures model performance subject to real-world perturbations that are likely to influence tokenization. Together, TokSuite allows robust decoupling of the influence of a model’s tokenizer, supporting a series of novel findings that elucidate the respective benefits and shortcomings of a wide range of popular tokenizers.

Method: Adversarial training framework with competitive dynamic between generator (user simulator) and discriminator, iteratively improving simulator realism through adversarial iterations.

Result: Fine-tuned simulators dramatically outperform zero-shot base models at surfacing system issues; adversarial training enhances diversity, distributional alignment, and predictive validity; strong correlation between simulated and real failure rates; discriminator accuracy drops drastically after three adversarial iterations.

Conclusion: Adversarial training is a promising approach for creating realistic user simulators in mental health support TOD domains, enabling rapid, reliable, and cost-effective system evaluation before deployment.

Abstract: Realistic user simulation is crucial for training and evaluating task-oriented dialogue (TOD) systems, yet creating simulators that accurately replicate human behavior remains challenging. A key property of effective simulators is their ability to expose failure modes of the systems they evaluate. We present an adversarial training framework that iteratively improves user simulator realism through a competitive dynamic between a generator (user simulator) and a discriminator. Applied to mental health support chatbots, our approach demonstrates that fine-tuned simulators dramatically outperform zero-shot base models at surfacing system issues, and adversarial training further enhances diversity, distributional alignment, and predictive validity. The resulting simulator achieves a strong correlation between simulated and real failure occurrence rates across diverse chatbot configurations while maintaining low distributional divergence of failure modes. Discriminator accuracy decreases drastically after three adversarial iterations, suggesting improved realism. These results provide evidence that adversarial training is a promising approach for creating realistic user simulators in mental health support TOD domains, enabling rapid, reliable, and cost-effective system evaluation before deployment.

Method: Controlled turn-level comparison where expert human tutors, novice human tutors, and multiple LLMs respond to the same math remediation conversation turns. Analysis of instructional strategies (restating/revoicing, pressing for accuracy) and linguistic characteristics (lexical diversity, readability, politeness, agency).

Result: LLMs approach expert levels of perceived pedagogical quality but show systematic differences: they underuse restating/revoicing strategies, produce longer, more lexically diverse, and more polite responses. Restating/revoicing, lexical diversity, and pressing for accuracy positively associate with pedagogical quality, while agentic/polite language negatively associate.

Conclusion: Recent LLMs exhibit pedagogical quality comparable to expert human tutors but rely on different instructional/linguistic strategies. Analysis of these strategies is valuable for evaluating tutoring responses across human tutors and intelligent tutoring systems.

Abstract: Recent work has explored the use of large language models for generating tutoring responses in mathematics, yet it remains unclear how closely their instructional behavior aligns with expert human practice. We examine this question using a controlled, turn-level comparison in which expert human tutors, novice human tutors, and multiple large language models respond to the same set of math remediation conversation turns. We examine both instructional strategies and linguistic characteristics of tutoring responses, including restating and revoicing, pressing for accuracy, lexical diversity, readability, politeness, and agency. We find that large language models approach expert levels of perceived pedagogical quality on average but exhibit systematic differences in their instructional and linguistic profiles. In particular, large language models tend to underuse restating and revoicing strategies characteristic of expert human tutors, while producing longer, more lexically diverse, and more polite responses. Statistical analyses show that restating and revoicing, lexical diversity, and pressing for accuracy are positively associated with perceived pedagogical quality, whereas higher levels of agentic and polite language are negatively associated. Overall, recent large language models exhibit levels of perceived pedagogical quality comparable to expert human tutors, while relying on different instructional and linguistic strategies. These findings underscore the value of analyzing instructional strategies and linguistic characteristics when evaluating tutoring responses across human tutors and intelligent tutoring systems.

Method: The authors explore model editing algorithms (ROME, IKE, WISE) and design new editing targets specifically for an unlearning setting, adapting these editing approaches to the task of information removal.

Result: Model editing approaches can exceed baseline unlearning methods in terms of quality of forgetting, depending on the specific setting. However, they still face challenges similar to traditional unlearning techniques.

Conclusion: While model editing algorithms show promise for unlearning tasks and can outperform existing methods, they still struggle to precisely define what should be unlearned without damaging overall model performance, indicating room for improvement in both approaches.

Abstract: Machine unlearning aims to remove unwanted information from a model, but many methods are inefficient for LLMs with large numbers of parameters or fail to fully remove the intended information without degrading performance on knowledge that should be retained. Model editing algorithms solve a similar problem of changing information in models, but they focus on redirecting inputs to a new target rather than removing that information altogether. In this work, we explore the editing algorithms ROME, IKE, and WISE and design new editing targets for an unlearning setting. Through this investigation, we show that model editing approaches can exceed baseline unlearning methods in terms of quality of forgetting depending on the setting. Like traditional unlearning techniques, they struggle to encapsulate the scope of what is to be unlearned without damage to the overall model performance.

Method: Proposed FEND framework integrates speech and text modalities, evaluated on 13 multi-lingual datasets across English, Chinese, Greek, French, and Dutch, with systematic multi-modal fusion analysis.

Result: Multi-modal fusion works well for AD and depression detection but underperforms for ASD due to dataset heterogeneity. Modality imbalance is common, and cross-corpus performance degrades in multi-lingual/task-heterogeneous settings.

Conclusion: FEND advances automated, lifespan-inclusive neuropsychiatric assessment and provides benchmarks for fair comparisons. Researchers are encouraged to adopt the framework for reproducible research.

Abstract: Neuropsychiatric disorders, such as Alzheimer’s disease (AD), depression, and autism spectrum disorder (ASD), are characterized by linguistic and acoustic abnormalities, offering potential biomarkers for early detection. Despite the promise of multi-modal approaches, challenges like multi-lingual generalization and the absence of a unified evaluation framework persist. To address these gaps, we propose FEND (Foundation model-based Evaluation of Neuropsychiatric Disorders), a comprehensive multi-modal framework integrating speech and text modalities for detecting AD, depression, and ASD across the lifespan. Leveraging 13 multi-lingual datasets spanning English, Chinese, Greek, French, and Dutch, we systematically evaluate multi-modal fusion performance. Our results show that multi-modal fusion excels in AD and depression detection but underperforms in ASD due to dataset heterogeneity. We also identify modality imbalance as a prevalent issue, where multi-modal fusion fails to surpass the best mono-modal models. Cross-corpus experiments reveal robust performance in task- and language-consistent scenarios but noticeable degradation in multi-lingual and task-heterogeneous settings. By providing extensive benchmarks and a detailed analysis of performance-influencing factors, FEND advances the field of automated, lifespan-inclusive, and multi-lingual neuropsychiatric disorder assessment. We encourage researchers to adopt the FEND framework for fair comparisons and reproducible research.

Method: Multi-task evaluation setup associating demographics with names, professions, and education levels. Comparison of attribution-based and correlation-based methods for locating bias features. Targeted sparse autoencoder feature ablations in Gemma-2-9B.

Result: Attribution-based ablations mitigate race and gender profession stereotypes while preserving name recognition accuracy. Correlation-based ablations are more effective for education bias. Removing attribution features in education tasks induces “prior collapse” and increases bias.

Conclusion: Demographic bias arises from task-specific mechanisms rather than absolute demographic markers. Mechanistic inference-time interventions enable surgical debiasing without compromising core model capabilities, highlighting need for dimension-specific interventions.

Abstract: We investigate how independent demographic bias mechanisms are from general demographic recognition in language models. Using a multi-task evaluation setup where demographics are associated with names, professions, and education levels, we measure whether models can be debiased while preserving demographic detection capabilities. We compare attribution-based and correlation-based methods for locating bias features. We find that targeted sparse autoencoder feature ablations in Gemma-2-9B reduce bias without degrading recognition performance: attribution-based ablations mitigate race and gender profession stereotypes while preserving name recognition accuracy, whereas correlation-based ablations are more effective for education bias. Qualitative analysis further reveals that removing attribution features in education tasks induces ``prior collapse’’, thus increasing overall bias. This highlights the need for dimension-specific interventions. Overall, our results show that demographic bias arises from task-specific mechanisms rather than absolute demographic markers, and that mechanistic inference-time interventions can enable surgical debiasing without compromising core model capabilities.

Method: Introduce semantic deceptions using novel symbols that replace standard digits and mathematical operators, then test LLMs’ ability to solve simple calculations in this altered notation while resisting misleading semantic cues.

Result: Semantic cues significantly deteriorate LLMs’ performance on simple tasks, revealing limitations in symbolic manipulation abilities and a tendency to over-rely on surface-level semantics, with chain-of-thought potentially amplifying statistical correlations.

Conclusion: Current LLMs have fundamental limitations in symbolic reasoning, raising ethical concerns about attributing reasoning abilities to them, particularly in decision-making contexts where robust abstraction is required and should not be compromised by residual semantic associations.

Abstract: Large language models (LLMs) are increasingly used in situations where human values are at stake, such as decision-making tasks that involve reasoning when performed by humans. We investigate the so-called reasoning capabilities of LLMs over novel symbolic representations by introducing an experimental framework that tests their ability to process and manipulate unfamiliar symbols. We introduce semantic deceptions: situations in which symbols carry misleading semantic associations due to their form, such as being embedded in specific contexts, designed to probe whether LLMs can maintain symbolic abstraction or whether they default to exploiting learned semantic associations. We redefine standard digits and mathematical operators using novel symbols, and task LLMs with solving simple calculations expressed in this altered notation. The objective is: (1) to assess LLMs’ capacity for abstraction and manipulation of arbitrary symbol systems; (2) to evaluate their ability to resist misleading semantic cues that conflict with the task’s symbolic logic. Through experiments with four LLMs we show that semantic cues can significantly deteriorate reasoning models’ performance on very simple tasks. They reveal limitations in current LLMs’ ability for symbolic manipulations and highlight a tendency to over-rely on surface-level semantics, suggesting that chain-of-thoughts may amplify reliance on statistical correlations. Even in situations where LLMs seem to correctly follow instructions, semantic cues still impact basic capabilities. These limitations raise ethical and societal concerns, undermining the widespread and pernicious tendency to attribute reasoning abilities to LLMs and suggesting how LLMs might fail, in particular in decision-making contexts where robust symbolic reasoning is essential and should not be compromised by residual semantic associations inherited from the model’s training.

Method: EssayCBM uses a rubric-aligned framework with eight dedicated prediction heads for specific writing concepts (Thesis Clarity, Evidence Use, etc.) on an encoder. These concept scores create a transparent bottleneck, and a lightweight network computes the final grade using only these concepts.

Result: EssayCBM matches the performance of black-box grading systems while providing actionable, concept-level feedback. The system allows instructors to adjust concept predictions and instantly see updated grades through an intuitive web interface.

Conclusion: EssayCBM successfully addresses the interpretability challenge in automated essay grading by providing transparent, concept-based assessment that enables accountable human-in-the-loop evaluation while maintaining competitive performance.

Abstract: Understanding how automated grading systems evaluate essays remains a significant challenge for educators and students, especially when large language models function as black boxes. We introduce EssayCBM, a rubric-aligned framework that prioritizes interpretability in essay assessment. Instead of predicting grades directly from text, EssayCBM evaluates eight writing concepts, such as Thesis Clarity and Evidence Use, through dedicated prediction heads on an encoder. These concept scores form a transparent bottleneck, and a lightweight network computes the final grade using only concepts. Instructors can adjust concept predictions and instantly view the updated grade, enabling accountable human-in-the-loop evaluation. EssayCBM matches black-box performance while offering actionable, concept-level feedback through an intuitive web interface.

Method: Introduce MediEval benchmark linking MIMIC-IV EHRs to unified knowledge base (UMLS/biomedical vocabularies). Generate diverse factual/counterfactual statements within real patient contexts. Use 4-quadrant framework evaluating knowledge grounding and contextual consistency. Propose Counterfactual Risk-Aware Fine-tuning (CoRFu) - DPO-based method with asymmetric penalty targeting unsafe confusions.

Result: Identified critical failure modes including hallucinated support and truth inversion that current LLMs frequently exhibit. CoRFu improves by +16.4 macro-F1 points over base model and eliminates truth inversion errors, demonstrating higher accuracy and substantially greater safety.

Conclusion: MediEval enables systematic evaluation of medical LLMs, revealing critical safety issues. CoRFu fine-tuning effectively addresses these risks, improving both accuracy and safety for medical applications.

Abstract: Large Language Models (LLMs) are increasingly applied to medicine, yet their adoption is limited by concerns over reliability and safety. Existing evaluations either test factual medical knowledge in isolation or assess patient-level reasoning without verifying correctness, leaving a critical gap. We introduce MediEval, a benchmark that links MIMIC-IV electronic health records (EHRs) to a unified knowledge base built from UMLS and other biomedical vocabularies. MediEval generates diverse factual and counterfactual medical statements within real patient contexts, enabling systematic evaluation across a 4-quadrant framework that jointly considers knowledge grounding and contextual consistency. Using this framework, we identify critical failure modes, including hallucinated support and truth inversion, that current proprietary, open-source, and domain-specific LLMs frequently exhibit. To address these risks, we propose Counterfactual Risk-Aware Fine-tuning (CoRFu), a DPO-based method with an asymmetric penalty targeting unsafe confusions. CoRFu improves by +16.4 macro-F1 points over the base model and eliminates truth inversion errors, demonstrating both higher accuracy and substantially greater safety.

Method: Developed a Mixture-of-Experts hybrid Mamba-Transformer architecture, pretrained on 25 trillion text tokens (including 3+ trillion new tokens over Nemotron 2), followed by supervised fine-tuning and large-scale reinforcement learning on diverse environments.

Result: Achieves better accuracy than Nemotron 2 Nano while activating less than half of parameters per forward pass, 3.3x higher inference throughput than similar open models (GPT-OSS-20B, Qwen3-30B-A3B-Thinking-2507), enhanced agentic, reasoning, and chat abilities, and supports up to 1M token context length.

Conclusion: Nemotron 3 Nano represents a significant advancement in efficient language modeling, offering superior performance with reduced computational requirements, making it suitable for practical applications requiring long context and high throughput.

Abstract: We present Nemotron 3 Nano 30B-A3B, a Mixture-of-Experts hybrid Mamba-Transformer language model. Nemotron 3 Nano was pretrained on 25 trillion text tokens, including more than 3 trillion new unique tokens over Nemotron 2, followed by supervised fine tuning and large-scale RL on diverse environments. Nemotron 3 Nano achieves better accuracy than our previous generation Nemotron 2 Nano while activating less than half of the parameters per forward pass. It achieves up to 3.3x higher inference throughput than similarly-sized open models like GPT-OSS-20B and Qwen3-30B-A3B-Thinking-2507, while also being more accurate on popular benchmarks. Nemotron 3 Nano demonstrates enhanced agentic, reasoning, and chat abilities and supports context lengths up to 1M tokens. We release both our pretrained Nemotron 3 Nano 30B-A3B Base and post-trained Nemotron 3 Nano 30B-A3B checkpoints on Hugging Face.

Method: Evaluate established retrieval strategies by measuring correlation between retrieval quality metrics and LLM-based judgments of response quality (responses generated from retrieved documents). Experiments across multiple datasets with low number of relevant documents (2-15). Introduce simple retrieval quality measure that doesn’t require knowledge of total relevant documents.

Result: The paper introduces a new retrieval quality measure that performs well without requiring knowledge of the total number of relevant documents, addressing the limitation of traditional recall-based metrics in realistic settings.

Conclusion: In practical retrieval scenarios where total relevant documents is unknown, the proposed simple measure provides effective evaluation without requiring recall computation, overcoming a fundamental limitation of traditional retrieval evaluation methods.

Abstract: In realistic retrieval settings with large and evolving knowledge bases, the total number of documents relevant to a query is typically unknown, and recall cannot be computed. In this paper, we evaluate several established strategies for handling this limitation by measuring the correlation between retrieval quality metrics and LLM-based judgments of response quality, where responses are generated from the retrieved documents. We conduct experiments across multiple datasets with a relatively low number of relevant documents (2-15). We also introduce a simple retrieval quality measure that performs well without requiring knowledge of the total number of relevant documents.

Method: Created “Hearing to Translate” test suite that comprehensively benchmarks 5 state-of-the-art SpeechLLMs against 16 strong direct and cascade systems. Evaluation spans 16 benchmarks, 13 language pairs, and 9 challenging conditions including disfluent, noisy, and long-form speech.

Result: Cascaded systems remain the most reliable overall. Current SpeechLLMs only match cascades in selected settings. Speech foundation models (SFMs) lag behind both approaches, showing that integrating an LLM (either within the model or in a pipeline) is essential for high-quality speech translation.

Conclusion: While SpeechLLMs represent an emerging approach, traditional cascade architectures combining speech foundation models with multilingual LLMs still provide the most reliable speech translation performance across diverse conditions.

Abstract: As Large Language Models (LLMs) expand beyond text, integrating speech as a native modality has given rise to SpeechLLMs, which aim to translate spoken language directly, thereby bypassing traditional transcription-based pipelines. Whether this integration improves speech-to-text translation quality over established cascaded architectures, however, remains an open question. We present Hearing to Translate, the first comprehensive test suite rigorously benchmarking 5 state-of-the-art SpeechLLMs against 16 strong direct and cascade systems that couple leading speech foundation models (SFM), with multilingual LLMs. Our analysis spans 16 benchmarks, 13 language pairs, and 9 challenging conditions, including disfluent, noisy, and long-form speech. Across this extensive evaluation, we find that cascaded systems remain the most reliable overall, while current SpeechLLMs only match cascades in selected settings and SFMs lag behind both, highlighting that integrating an LLM, either within the model or in a pipeline, is essential for high-quality speech translation.

Method: Uses Mixture-of-Experts hybrid Mamba-Transformer architecture with LatentMoE (novel approach for model quality improvement), NVFP4 training for Super/Ultra models, MTP layers for faster text generation, and multi-environment reinforcement learning post-training.

Result: Nano outperforms comparable models in accuracy while remaining cost-efficient; Super is optimized for collaborative agents and IT automation; Ultra provides state-of-the-art accuracy and reasoning performance; all models support up to 1M token context.

Conclusion: Nemotron 3 family offers a scalable solution with different model sizes optimized for various applications, with open release of weights, software, recipes, and data, making advanced AI capabilities accessible for different computational budgets and use cases.

Abstract: We introduce the Nemotron 3 family of models - Nano, Super, and Ultra. These models deliver strong agentic, reasoning, and conversational capabilities. The Nemotron 3 family uses a Mixture-of-Experts hybrid Mamba-Transformer architecture to provide best-in-class throughput and context lengths of up to 1M tokens. Super and Ultra models are trained with NVFP4 and incorporate LatentMoE, a novel approach that improves model quality. The two larger models also include MTP layers for faster text generation. All Nemotron 3 models are post-trained using multi-environment reinforcement learning enabling reasoning, multi-step tool use, and support granular reasoning budget control. Nano, the smallest model, outperforms comparable models in accuracy while remaining extremely cost-efficient for inference. Super is optimized for collaborative agents and high-volume workloads such as IT ticket automation. Ultra, the largest model, provides state-of-the-art accuracy and reasoning performance. Nano is released together with its technical report and this white paper, while Super and Ultra will follow in the coming months. We will openly release the model weights, pre- and post-training software, recipes, and all data for which we hold redistribution rights.

Method: Progressive architectural study starting from linear next-token predictors, adding nonlinearities, self-attention, and multi-layer transformers. Evaluated on character-level modeling (Tiny Shakespeare) and word-level modeling (PTB, WikiText-2). Compared models using test NLL, parameter count, and training FLOPs.

Result: Attention-based models dominate MLPs in per-FLOP efficiency even at small scale. Increasing depth or context without sufficient optimization can degrade performance. Rotary positional embeddings (successful in large models) don’t necessarily transfer to small-model regimes.

Conclusion: Small language models require different architectural considerations than large models, with attention mechanisms showing superior efficiency but some large-model techniques failing to transfer effectively to constrained compute settings.

Abstract: We present a systematic empirical study of small language models under strict compute constraints, analyzing how architectural choices and training budget interact to determine performance. Starting from a linear next-token predictor, we progressively introduce nonlinearities, self-attention, and multi-layer transformer architectures, evaluating each on character-level modeling of Tiny Shakespeare and word-level modeling of Penn Treebank (PTB) and WikiText-2. We compare models using test negative log-likelihood (NLL), parameter count, and approximate training FLOPs to characterize accuracy-efficiency trade-offs. Our results show that attention-based models dominate MLPs in per-FLOP efficiency even at small scale, while increasing depth or context without sufficient optimization can degrade performance. We further examine rotary positional embeddings (RoPE), finding that architectural techniques successful in large language models do not necessarily transfer to small-model regimes.

Method: Cross-model Reasoning Distillation Provenance Tracing framework that compares predictive probabilities from teacher, original student, and distilled models to classify each action’s origin, plus teacher-guided data selection based on teacher-student divergences.

Result: Distilled models can generate teacher-originated actions in test-time contexts, correlating with observed performance. The teacher-guided data selection method proves effective across multiple teacher and student models.

Conclusion: The provenance-tracing framework provides insights into reasoning distillation and shows promise for improving distillation methods, with teacher-guided data selection offering a principled approach over heuristic methods.

Abstract: Reasoning distillation has attracted increasing attention. It typically leverages a large teacher model to generate reasoning paths, which are then used to fine-tune a student model so that it mimics the teacher’s behavior in training contexts. However, previous approaches have lacked a detailed analysis of the origins of the distilled model’s capabilities. It remains unclear whether the student can maintain consistent behaviors with the teacher in novel test-time contexts, or whether it regresses to its original output patterns, raising concerns about the generalization of distillation models. To analyse this question, we introduce a cross-model Reasoning Distillation Provenance Tracing framework. For each action (e.g., a sentence) produced by the distilled model, we obtain the predictive probabilities assigned by the teacher, the original student, and the distilled model under the same context. By comparing these probabilities, we classify each action into different categories. By systematically disentangling the provenance of each action, we experimentally demonstrate that, in test-time contexts, the distilled model can indeed generate teacher-originated actions, which correlate with and plausibly explain observed performance on distilled model. Building on this analysis, we further propose a teacher-guided data selection method. Unlike prior approach that rely on heuristics, our method directly compares teacher-student divergences on the training data, providing a principled selection criterion. We validate the effectiveness of our approach across multiple representative teacher models and diverse student models. The results highlight the utility of our provenance-tracing framework and underscore its promise for reasoning distillation. We hope to share Reasoning Distillation Provenance Tracing and our insights into reasoning distillation with the community.

Method: Propose a neural network framework for token-level hallucination detection. Freeze LLM parameters and use lightweight MLP probes for nonlinear modeling of high-level hidden states. Design multi-objective joint loss function for detection stability and semantic disambiguity. Establish layer position-probe performance response model using Bayesian optimization to automatically search for optimal probe insertion layers.

Result: Experimental results on LongFact, HealthBench, and TriviaQA show MLP probes significantly outperform state-of-the-art methods in accuracy, recall, and detection capability under low false-positive conditions.

Conclusion: The proposed MLP probe framework effectively detects hallucinations in LLMs by capturing nonlinear semantic structures, offering real-time, lightweight detection that surpasses existing methods across multiple benchmarks.

Abstract: Large language models(LLMs) excel at text generation and knowledge question-answering tasks, but they are prone to generating hallucinated content, severely limiting their application in high-risk domains. Current hallucination detection methods based on uncertainty estimation and external knowledge retrieval suffer from the limitation that they still produce erroneous content at high confidence levels and rely heavily on retrieval efficiency and knowledge coverage. In contrast, probe methods that leverage the model’s hidden-layer states offer real-time and lightweight advantages. However, traditional linear probes struggle to capture nonlinear structures in deep semantic spaces.To overcome these limitations, we propose a neural network-based framework for token-level hallucination detection. By freezing language model parameters, we employ lightweight MLP probes to perform nonlinear modeling of high-level hidden states. A multi-objective joint loss function is designed to enhance detection stability and semantic disambiguity. Additionally, we establish a layer position-probe performance response model, using Bayesian optimization to automatically search for optimal probe insertion layers and achieve superior training results.Experimental results on LongFact, HealthBench, and TriviaQA demonstrate that MLP probes significantly outperform state-of-the-art methods in accuracy, recall, and detection capability under low false-positive conditions.

Method: TriAligner uses dual-encoder architecture with contrastive learning, incorporates both native and English translations across modalities, employs data preprocessing/augmentation with LLMs, and uses hard negative sampling for representation learning.

Result: Significant improvements in retrieval accuracy and fact-checking performance over baselines on both monolingual and crosslingual benchmarks.

Conclusion: The proposed TriAligner system effectively addresses multilingual fact-checked claim retrieval by learning source importance in alignment and demonstrating robust performance across language settings.

Abstract: This paper presents our system for SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval. In an era where misinformation spreads rapidly, effective fact-checking is increasingly critical. We introduce TriAligner, a novel approach that leverages a dual-encoder architecture with contrastive learning and incorporates both native and English translations across different modalities. Our method effectively retrieves claims across multiple languages by learning the relative importance of different sources in alignment. To enhance robustness, we employ efficient data preprocessing and augmentation using large language models while incorporating hard negative sampling to improve representation learning. We evaluate our approach on monolingual and crosslingual benchmarks, demonstrating significant improvements in retrieval accuracy and fact-checking performance over baselines.

Method: Proposes reflection pretraining that enables biological sequence models to generate auxiliary “thinking tokens” beyond answer tokens. Introduces concept of language expressiveness and augments token set to enhance reasoning capacity.

Result: Theoretically shows augmented token set enhances biological language expressiveness. Experimentally demonstrates that reflection pretraining teaches protein models to self-correct and achieves substantial performance gains over standard pretraining.

Conclusion: Reflection pretraining successfully enables CoT-style reasoning in biological sequence models by overcoming token expressiveness limitations, leading to improved reasoning and performance.

Abstract: Chain-of-Thought (CoT) prompting has significantly advanced task-solving capabilities in natural language processing with large language models. Unlike standard prompting, CoT encourages the model to generate intermediate reasoning steps, non-answer tokens, that help guide the model toward more accurate final outputs. These intermediate steps enable more complex reasoning processes such as error correction, memory management, future planning, and self-reflection. However, applying CoT to non-natural language domains, such as protein and RNA language models, is not yet possible, primarily due to the limited expressiveness of their token spaces (e.g., amino acid tokens). In this work, we propose and define the concept of language expressiveness: the ability of a given language, using its tokens and grammar, to encode information. We show that the limited expressiveness of protein language severely restricts the applicability of CoT-style reasoning. To overcome this, we introduce reflection pretraining, for the first time in a biological sequence model, which enables the model to engage in intermediate reasoning through the generation of auxiliary “thinking tokens” beyond simple answer tokens. Theoretically, we demonstrate that our augmented token set significantly enhances biological language expressiveness, thereby improving the overall reasoning capacity of the model. Experimentally, our pretraining approach teaches protein models to self-correct and leads to substantial performance gains compared to standard pretraining.

Method: Three-step pipeline: (1) creates complex conversational data between LLM patient and LLM doctor, (2) evaluates conversations with a committee of 2 LLM judges across multiple dimensions, (3) converts identified mistakes into simplified single-shot QA scenarios.

Result: Created MedMistake-All dataset of 3,390 QA pairs where GPT-5 and Gemini 2.5 Pro fail. Validated subset of 211 questions (MedMistake-Bench) with medical experts. Evaluation of 12 frontier LLMs showed GPT models, Claude, and Grok performed best on the benchmark.

Conclusion: MedMistake provides an automated pipeline for identifying and benchmarking LLM mistakes in clinical reasoning, with released datasets enabling standardized evaluation of medical LLM performance across different models.

Abstract: Large language models (LLMs) are increasingly evaluated in clinical settings using multi-dimensional rubrics which quantify reasoning quality, safety, and patient-centeredness. Yet, replicating specific mistakes in other LLM models is not straightforward and often requires manual effort. We introduce MedMistake, an automatic pipeline that extracts mistakes LLMs make in patient-doctor conversations and converts them into a benchmark of single-shot QA pairs. Our pipeline (1) creates complex, conversational data between an LLM patient and LLM doctor, (2) runs an evaluation with a committee of 2 LLM judges across a variety of dimensions and (3) creates simplified single-shot QA scenarios from those mistakes. We release MedMistake-All, a dataset of 3,390 single-shot QA pairs where GPT-5 and Gemini 2.5 Pro are currently failing to answer correctly, as judged by two LLM judges. We used medical experts to validate a subset of 211/3390 questions (MedMistake-Bench), which we used to run a final evaluation of 12 frontier LLMs: Claude Opus 4.5, Claude Sonnet 4.5, DeepSeek-Chat, Gemini 2.5 Pro, Gemini 3 Pro, GPT-4o, GPT-5, GPT-5.1, GPT-5.2, Grok 4, Grok 4.1, Mistral Large. We found that GPT models, Claude and Grok obtained the best performance on MedMistake-Bench. We release both the doctor-validated benchmark (MedMistake-Bench), as well as the full dataset (MedMistake-All) at https://huggingface.co/datasets/TheLumos/MedicalMistakeBenchmark.

Method: Analyze how supervision allocation across prompt (P), chain-of-thought (CoT), and answer (A) segments affects distillation. Develop selective knowledge distillation focusing only on CoT tokens when prompt and answer information is encompassed by them. Establish a truncation protocol to quantify computation-quality tradeoffs as a function of sequence length.

Result: Training on only the first 50% of tokens retains ≈94% of full-sequence performance on math benchmarks while reducing training time, memory usage, and FLOPs by about 50% each. Selective distillation over CoT tokens is effective when prompt and answer information is encompassed within the reasoning.

Conclusion: Reasoning distillation benefits from prioritizing early reasoning tokens, providing a simple lever for computation-quality tradeoffs. This approach enables efficient knowledge transfer from large to small models while maintaining strong performance.

Abstract: Distilling the reasoning capabilities from a large language model (LLM) to a smaller student model often involves training on substantial amounts of reasoning data. However, distillation over lengthy sequences with prompt (P), chain-of-thought (CoT), and answer (A) segments makes the process computationally expensive. In this work, we investigate how the allocation of supervision across different segments (P, CoT, A) affects student performance. Our analysis shows that selective knowledge distillation over only the CoT tokens can be effective when the prompt and answer information is encompassed by it. Building on this insight, we establish a truncation protocol to quantify computation-quality tradeoffs as a function of sequence length. We observe that training on only the first $50%$ of tokens of every training sequence can retain, on average, $\approx94%$ of full-sequence performance on math benchmarks while reducing training time, memory usage, and FLOPs by about $50%$ each. These findings suggest that reasoning distillation benefits from prioritizing early reasoning tokens and provides a simple lever for computation-quality tradeoffs. Codes are available at https://github.com/weiruichen01/distilling-the-essence.

Method: SFTKey uses a two-stage training scheme: Stage 1 applies conventional SFT to ensure proper output format, while Stage 2 fine-tunes only the Key portion (final answer) to improve accuracy.

Result: Extensive experiments across multiple benchmarks and model families show SFTKey achieves average accuracy improvement exceeding 5% over conventional SFT while preserving correct format generation ability.

Conclusion: This study advances LLM fine-tuning by explicitly balancing CoT learning with additional optimization on answer-relevant tokens, addressing attention allocation issues in complex reasoning tasks.

Abstract: With the rapid advancement of Large Language Models (LLMs), the Chain-of-Thought (CoT) component has become significant for complex reasoning tasks. However, in conventional Supervised Fine-Tuning (SFT), the model could allocate disproportionately more attention to CoT sequences with excessive length. This reduces focus on the much shorter but essential Key portion-the final answer, whose correctness directly determines task success and evaluation quality. To address this limitation, we propose SFTKey, a two-stage training scheme. In the first stage, conventional SFT is applied to ensure proper output format, while in the second stage, only the Key portion is fine-tuned to improve accuracy. Extensive experiments across multiple benchmarks and model families demonstrate that SFTKey achieves an average accuracy improvement exceeding 5% over conventional SFT, while preserving the ability to generate correct formats. Overall, this study advances LLM fine-tuning by explicitly balancing CoT learning with additional optimization on answer-relevant tokens.

Method: Data-Adaptive Semantic Refinement (DAS) framework couples fixed GNN with LLM in closed feedback loop. GNN provides supervisory signals to guide LLM’s semantic refinement, refined semantics update the graph learner.

Result: Consistent improvements on structure-dominated graphs while remaining competitive on semantics-rich graphs across both text-rich and text-free graphs.

Conclusion: Data-centric semantic adaptation effectively addresses structure-semantics heterogeneity in graphs through adaptive semantic refinement rather than fixed model biases.

Abstract: Graph-structured data exhibit substantial heterogeneity in where their predictive signals originate: in some domains, node-level semantics dominate, while in others, structural patterns play a central role. This structure-semantics heterogeneity implies that no graph learning model with a fixed inductive bias can generalize optimally across diverse graph domains. However, most existing methods address this challenge from the model side by incrementally injecting new inductive biases, which remains fundamentally limited given the open-ended diversity of real-world graphs. In this work, we take a data-centric perspective and treat node semantics as a task-adaptive variable. We propose a Data-Adaptive Semantic Refinement framework DAS for graph representation learning, which couples a fixed graph neural network (GNN) and a large language model (LLM) in a closed feedback loop. The GNN provides implicit supervisory signals to guide the semantic refinement of LLM, and the refined semantics are fed back to update the same graph learner. We evaluate our approach on both text-rich and text-free graphs. Results show consistent improvements on structure-dominated graphs while remaining competitive on semantics-rich graphs, demonstrating the effectiveness of data-centric semantic adaptation under structure-semantics heterogeneity.

Method: Proposes using semi-supervised learning techniques that leverage both labeled and unlabeled data, with emphasis on task-specific augmentations rather than general-purpose ones.

Result: Semi-supervised learning improves safety classification performance for both LLM prompts and responses, with task-specific augmentations significantly outperforming general-purpose augmentation techniques.

Conclusion: Semi-supervised learning with task-specific augmentations offers an effective alternative to traditional supervised approaches for LLM safety classification, addressing data acquisition challenges while improving performance.

Abstract: Safety for Large Language Models (LLMs) has been an ongoing research focus since their emergence and is even more relevant nowadays with the increasing capacity of those models. Currently, there are several guardrails in place for all public LLMs and multiple proposed datasets for training safety classifiers. However, training these safety classifiers relies on large quantities of labeled data, which can be problematic to acquire, prone to labeling errors, or often include synthetic data. To address these issues, we suggest a different approach: utilizing semi-supervised learning techniques, which leverage both labeled and unlabeled data, to improve the performance on the safety task. We analyze the improvements that these techniques can offer for both prompts given to Large Language Models and the responses to those requests. Moreover, since augmentation is the central part of semi-supervised algorithms, we demonstrate the importance of using task-specific augmentations, which significantly increase the performance when compared to general-purpose augmentation techniques.

Method: Created ClarifyMT-Bench using a five-dimensional ambiguity taxonomy and six behaviorally diverse simulated user personas. Constructed 6,120 multi-turn dialogues through hybrid LLM-human pipeline. Evaluated ten representative LLMs and proposed ClarifyAgent approach that decomposes clarification into perception, forecasting, tracking, and planning components.

Result: LLMs show consistent under-clarification bias - they tend to answer prematurely, and performance degrades as dialogue depth increases. ClarifyAgent substantially improves robustness across ambiguity conditions compared to baseline LLMs.

Conclusion: ClarifyMT-Bench provides a reproducible foundation for studying when LLMs should ask vs. answer questions and how to navigate ambiguity in real-world human-LLM interactions, with ClarifyAgent demonstrating improved clarification capabilities.

Abstract: Large language models (LLMs) are increasingly deployed as conversational assistants in open-domain, multi-turn settings, where users often provide incomplete or ambiguous information. However, existing LLM-focused clarification benchmarks primarily assume single-turn interactions or cooperative users, limiting their ability to evaluate clarification behavior in realistic settings. We introduce \textbf{ClarifyMT-Bench}, a benchmark for multi-turn clarification grounded in a five-dimensional ambiguity taxonomy and a set of six behaviorally diverse simulated user personas. Through a hybrid LLM-human pipeline, we construct 6,120 multi-turn dialogues capturing diverse ambiguity sources and interaction patterns. Evaluating ten representative LLMs uncovers a consistent under-clarification bias: LLMs tend to answer prematurely, and performance degrades as dialogue depth increases. To mitigate this, we propose \textbf{ClarifyAgent}, an agentic approach that decomposes clarification into perception, forecasting, tracking, and planning, substantially improving robustness across ambiguity conditions. ClarifyMT-Bench establishes a reproducible foundation for studying when LLMs should ask, when they should answer, and how to navigate ambiguity in real-world human-LLM interactions.

Method: 1) Cast low-resource speech representation learning as meta-learning problem; 2) Multi-task adaptive pre-training (MAdaPT) protocol with bi-level optimization; 3) First-order bi-level optimization (FOBLO) for scalable meta-training; 4) Stabilization via interleaved supervision alternating self-supervised and supervised objectives.

Result: SpidR-Adapt achieves rapid gains in phonemic discriminability (ABX) and spoken language modeling (sWUGGY, sBLIMP, tSC), outperforming in-domain language models after training on less than 1 hour of target-language audio, making it over 100x more data-efficient than standard training.

Conclusion: The approach provides a practical, architecture-agnostic path toward biologically inspired, data-efficient speech representations, bridging the gap between human language acquisition efficiency and machine learning models.

Abstract: Human infants, with only a few hundred hours of speech exposure, acquire basic units of new languages, highlighting a striking efficiency gap compared to the data-hungry self-supervised speech models. To address this gap, this paper introduces SpidR-Adapt for rapid adaptation to new languages using minimal unlabeled data. We cast such low-resource speech representation learning as a meta-learning problem and construct a multi-task adaptive pre-training (MAdaPT) protocol which formulates the adaptation process as a bi-level optimization framework. To enable scalable meta-training under this framework, we propose a novel heuristic solution, first-order bi-level optimization (FOBLO), avoiding heavy computation costs. Finally, we stabilize meta-training by using a robust initialization through interleaved supervision which alternates self-supervised and supervised objectives. Empirically, SpidR-Adapt achieves rapid gains in phonemic discriminability (ABX) and spoken language modeling (sWUGGY, sBLIMP, tSC), improving over in-domain language models after training on less than 1h of target-language audio, over $100\times$ more data-efficient than standard training. These findings highlight a practical, architecture-agnostic path toward biologically inspired, data-efficient representations. We open-source the training code and model checkpoints at https://github.com/facebookresearch/spidr-adapt.

Method: SMART uses a hierarchical approach with three components: (1) Grammarian Tree LSTM for syntax-aware fact extraction as subject-relation-object triples, (2) compact indexed MANN memory storing facts as 384D vectors with source information, and (3) 6-layer Transformer for fusing retrieved facts into responses.

Result: SMART uses only 45.51M parameters (64% less than GPT-2, 69% less than BERT) and achieves 21.3% higher accuracy than GPT-2. It supports dual inference modes: fast path for known documents (sub-second answers) and dynamic path with RAG for new uploads.

Conclusion: SMART provides a practical solution for engineering manual processing, delivering more well-supported results with reduced hallucinations compared to small transformer models, while being more efficient and accurate.

Abstract: The user of Engineering Manuals (EM) finds it difficult to read EM s because they are long, have a dense format which includes written documents, step by step procedures, and standard parameter lists for engineering equipment. Off the shelf transformers, especially compact ones, treat this material as a flat stream of tokens. This approach leads to confident but incorrect numeric answers and forces the models to memorize separate facts inefficiently. SMART (Structured Memory and Reasoning Transformer) offers a different and practical solution to the above problem. SMART structures its processing by using a hierarchical approach, and is based upon three main job categories (1) A syntax-aware Fact Extractor (Grammarian) Tree LSTM which extracts facts as subject relation object relations from EM sentences (2) A compact indexed memory MANN (Memory Augmented Neural Network) that indexes these Rational Subject Relation Objects as 384 dimensional vectors that are associated with the source of the information, and (3) A 6 layer Transformer that learns to fuse the previously retrieved facts into its generated response. The entire SMART model utilizes 45.51M parameters, which is 64% less than GPT-2 (124M) and 69% less than BERT (133M), and it achieves a 21.3% higher accuracy than GPT-2, indicating that SMART fits the data better with the least amount of processing requirements. SMART employs dual modes of inference an indexed fast path for known documents (sub-second answer times) and an indexed dynamic path assisted by RAGs for new uploads (FAISS Top 20 results with memory severed at 64 slots). In real world deployment, this framework leads to more well supported results with reduced hallucinations than comparable small transformer models.

Method: PTP incorporates sampling procedure into the model to jointly predict multiple dependent tokens in one transformer call, trained via distillation or inverse autoregressive training without a teacher.

Result: Achieves state-of-the-art speculative decoding performance on Vicuna-7B, accepting over four tokens per step on Spec-Bench while maintaining ability to represent arbitrary autoregressive distributions.

Conclusion: Parallel generation of long sequences is feasible without loss of modeling power, as demonstrated by PTP’s universal framework for parallel sequence generation.

Abstract: We propose Parallel Token Prediction (PTP), a universal framework for parallel sequence generation in language models. PTP jointly predicts multiple dependent tokens in a single transformer call by incorporating the sampling procedure into the model. This reduces the latency bottleneck of autoregressive decoding, and avoids the restrictive independence assumptions common in existing multi-token prediction methods. We prove that PTP can represent arbitrary autoregressive sequence distributions. PTP is trained either by distilling an existing model or through inverse autoregressive training without a teacher. Experimentally, we achieve state-of-the-art speculative decoding performance on Vicuna-7B by accepting over four tokens per step on Spec-Bench. The universality of our framework indicates that parallel generation of long sequences is feasible without loss of modeling power.

Method: The authors introduce a two-stage experimental pipeline that explicitly separates perception from reasoning: (1) Perception stage: each image is independently converted into natural-language descriptions, (2) Reasoning stage: a model induces and applies rules using these descriptions. This design prevents cross-image inductive signal leakage and isolates reasoning from perception bottlenecks. They test this pipeline across three ARC-style datasets: Mini-ARC, ACRE, and Bongard-LOGO, comparing it against standard end-to-end evaluation.

Result: Results show that perception capability is the dominant factor underlying the observed performance gap. Manual inspection of reasoning traces reveals that approximately 80% of model failures stem from perception errors rather than reasoning errors. The two-stage pipeline demonstrates that ARC-style benchmarks conflate perceptual and reasoning challenges.

Conclusion: The conclusion is that ARC-style benchmarks conflate perception and reasoning challenges, and observed performance gaps may overstate deficiencies in machine reasoning. The findings underscore the need for evaluation protocols that disentangle perception from reasoning when assessing progress in machine intelligence.

Abstract: Reasoning benchmarks such as the Abstraction and Reasoning Corpus (ARC) and ARC-AGI are widely used to assess progress in artificial intelligence and are often interpreted as probes of core, so-called ``fluid’’ reasoning abilities. Despite their apparent simplicity for humans, these tasks remain challenging for frontier vision-language models (VLMs), a gap commonly attributed to deficiencies in machine reasoning. We challenge this interpretation and hypothesize that the gap arises primarily from limitations in visual perception rather than from shortcomings in inductive reasoning. To verify this hypothesis, we introduce a two-stage experimental pipeline that explicitly separates perception and reasoning. In the perception stage, each image is independently converted into a natural-language description, while in the reasoning stage a model induces and applies rules using these descriptions. This design prevents leakage of cross-image inductive signals and isolates reasoning from perception bottlenecks. Across three ARC-style datasets, Mini-ARC, ACRE, and Bongard-LOGO, we show that the perception capability is the dominant factor underlying the observed performance gap by comparing the two-stage pipeline with against standard end-to-end one-stage evaluation. Manual inspection of reasoning traces in the VLM outputs further reveals that approximately 80 percent of model failures stem from perception errors. Together, these results demonstrate that ARC-style benchmarks conflate perceptual and reasoning challenges and that observed performance gaps may overstate deficiencies in machine reasoning. Our findings underscore the need for evaluation protocols that disentangle perception from reasoning when assessing progress in machine intelligence.

Method: Builds on Qwen-2.5-Coder backbones, uses Pooling by Multihead Attention (PMA) module to generate sequence embeddings from token embeddings, trained on 3 million publicly available data.

Result: C2LLM sets new records on MTEB-Code among similar-sized models, with C2LLM-7B ranking 1st on the overall leaderboard.

Conclusion: C2LLM demonstrates superior code embedding capabilities through its PMA approach, effectively leveraging pretrained LLM representations while providing flexible and high-quality sequence embeddings for code.

Abstract: We present C2LLM - Contrastive Code Large Language Models, a family of code embedding models in both 0.5B and 7B sizes. Building upon Qwen-2.5-Coder backbones, C2LLM adopts a Pooling by Multihead Attention (PMA) module for generating sequence embedding from token embeddings, effectively 1) utilizing the LLM’s causal representations acquired during pretraining, while also 2) being able to aggregate information from all tokens in the sequence, breaking the information bottleneck in EOS-based sequence embeddings, and 3) supporting flexible adaptation of embedding dimension, serving as an alternative to MRL. Trained on three million publicly available data, C2LLM models set new records on MTEB-Code among models of similar sizes, with C2LLM-7B ranking 1st on the overall leaderboard.

Method: The paper introduces Denoising Entropy, a computable metric that quantifies cumulative predictive uncertainty along generative paths. Using this metric, they propose two algorithms: a post-hoc selection method and a real-time guidance strategy to optimize decoding paths.

Result: Experiments show that entropy-guided methods significantly improve generation quality, consistently boosting accuracy on challenging reasoning, planning, and code benchmarks.

Conclusion: The work establishes Denoising Entropy as a principled tool for understanding and controlling generation in MDMs, effectively turning uncertainty from a liability into a key advantage for discovering high-quality solutions.

Abstract: Masked Diffusion Models (MDMs) offer flexible, non-autoregressive generation, but this freedom introduces a challenge: final output quality is highly sensitive to the decoding order. We are the first to formalize this issue, attributing the variability in output quality to the cumulative predictive uncertainty along a generative path. To quantify this uncertainty, we introduce Denoising Entropy, a computable metric that serves as an internal signal for evaluating generative process. Leveraging this metric, we propose two algorithms designed to optimize the decoding path: a post-hoc selection method and a real-time guidance strategy. Experiments demonstrate that our entropy-guided methods significantly improve generation quality, consistently boosting accuracy on challenging reasoning, planning, and code benchmarks. Our work establishes Denoising Entropy as a principled tool for understanding and controlling generation, effectively turning the uncertainty in MDMs from a liability into a key advantage for discovering high-quality solutions.

Method: Proposed method incorporates world knowledge features including linked entities and fine-grained entity types into a neural question generation model to encode additional entity-related information from passages.

Result: The world knowledge enriched model outperforms vanilla neural question generation by 1.37 and 1.59 absolute BLEU 4 scores on SQuAD and MS MARCO test datasets respectively.

Conclusion: Incorporating world knowledge features significantly improves question generation performance, demonstrating the usefulness of entity information for generating more human-like questions.

Abstract: In this paper, we propose a method for incorporating world knowledge (linked entities and fine-grained entity types) into a neural question generation model. This world knowledge helps to encode additional information related to the entities present in the passage required to generate human-like questions. We evaluate our models on both SQuAD and MS MARCO to demonstrate the usefulness of the world knowledge features. The proposed world knowledge enriched question generation model is able to outperform the vanilla neural question generation model by 1.37 and 1.59 absolute BLEU 4 score on SQuAD and MS MARCO test dataset respectively.

Method: Replaces binary MLE reward modeling with sequence MLE, enabling richer language feedback without additional annotations, models, or training stages. Uses seq2seq approach for reward modeling.

Result: Reduces refusal-to-response in safety dialogues and long-response bias in summarization. Improves RLHF performance across 2B and 7B LLMs on 3 NLP tasks with average 76.9% win rate. Works under out-of-distribution prompts.

Conclusion: Seq2seq reward modeling effectively mitigates RLHF biases by providing richer language feedback, improving alignment without additional annotation costs.

Abstract: Aligning the behavior of Large language models (LLMs) with human intentions and values remains a critical challenge. Reinforcement learning from human feedback (RLHF) aligns LLMs by training a reward model (RM) on human preferences and fine-tuning the LLMs to maximize RM feedback. Despite its effectiveness and popularity, RLHF is prone to biased local optimization. It means RM fails to provide feedback that accurately aligns with human preference, causing LLMs to explore unexpected generalizations, and failing to achieve alignment objectives. To mitigate this issue, we propose a novel \textit{sequence-to-sequence (seq2seq) reward modeling} method. Its key insight is that learning from language feedback rather than scalar feedback improves RLHF without additional annotations. We replaced the reward modeling target from binary maximum likelihood estimation (MLE) with sequence MLE. This method enables richer and fine-grained language feedback without additional annotations, models, or training stages. Our experiments demonstrated its effectiveness, specifically, reducing the refusal-to-response paradigm in single-turn safety dialogues and the long-response bias in text summarization tasks. We provide further analysis that seq2seq RM improves RLHF performance across 2B and 7B LLMs on 3 NLP tasks, achieving an average win rate of 76.9%. We further show that seq2seq RM can still improve the performance of RLHF under out-of-distribution prompts.

Method: CAKE frames KV cache eviction as a “cake-slicing problem,” assessing layer-specific preferences using attention dynamics in spatial and temporal dimensions, allocating cache sizes accordingly, and managing memory constraints in a cascading manner with a new eviction indicator that tracks token importance shifts over time.

Result: CAKE maintains model performance with only 3.2% of KV cache, consistently outperforms current baselines across various models and memory constraints (especially in low-memory settings), and achieves over 10x speedup in decoding latency for 128K token contexts with FlashAttention-2.

Conclusion: CAKE provides an effective solution for KV cache management that adaptively allocates resources across layers while maintaining memory budgets, significantly improving inference efficiency without sacrificing model performance.

Abstract: Large language models (LLMs) excel at processing long sequences, boosting demand for key-value (KV) caching. While recent efforts to evict KV cache have alleviated the inference burden, they often fail to allocate resources rationally across layers with different attention patterns. In this paper, we introduce Cascading and Adaptive KV cache Eviction (CAKE), a novel approach that frames KV cache eviction as a “cake-slicing problem.” CAKE assesses layer-specific preferences by considering attention dynamics in both spatial and temporal dimensions, allocates rational cache size for layers accordingly, and manages memory constraints in a cascading manner. This approach enables a global view of cache allocation, adaptively distributing resources across diverse attention mechanisms while maintaining memory budgets. CAKE also employs a new eviction indicator that considers the shifting importance of tokens over time, addressing limitations in existing methods that overlook temporal dynamics. Comprehensive experiments on LongBench and NeedleBench show that CAKE maintains model performance with only 3.2% of the KV cache and consistently outperforms current baselines across various models and memory constraints, particularly in low-memory settings. Additionally, CAKE achieves over 10x speedup in decoding latency compared to full cache when processing contexts of 128K tokens with FlashAttention-2. Our code is available at https://github.com/antgroup/cakekv.

Method: Two key strategies: (1) Fine-tune a compact 8B-parameter model augmented with teacher-generated reasoning traces for efficient real-time breakdown detection and explanation. (2) Systematically evaluate frontier LLMs using advanced prompting (few-shot, chain-of-thought, analogical reasoning) for high-fidelity breakdown assessment, integrated into an “escalation” architecture where the efficient detector defers to larger models only when necessary.

Result: The fine-tuned model demonstrates robust classification and calibration on English and Japanese dialogues, generalizes to BETOLD dataset with 7% accuracy improvement over baseline. Achieves state-of-the-art performance on DBDC5 and strong results on BETOLD, outperforming specialized classifiers on DBDC5 and narrowing performance gap to larger proprietary models. The monitor-escalate pipeline reduces inference costs by 54%.

Conclusion: The proposed framework provides a cost-effective and interpretable solution for robust conversational AI in high-impact domains by efficiently managing dialogue breakdowns while substantially reducing operational costs and computational overhead.

Abstract: Large Language Models (LLMs) have demonstrated substantial capabilities in conversational AI applications, yet their susceptibility to dialogue breakdowns poses significant challenges to deployment reliability and user trust. This paper introduces a “Detect, Explain, Escalate” framework to manage dialogue breakdowns in LLM-powered agents, emphasizing resource-efficient operation. Our approach integrates two key strategies: (1) We fine-tune a compact 8B-parameter model, augmented with teacher-generated reasoning traces, which serves as an efficient real-time breakdown detector and explainer. This model demonstrates robust classification and calibration on English and Japanese dialogues, and generalizes to the BETOLD dataset, improving accuracy by 7% over its baseline. (2) We systematically evaluate frontier LLMs using advanced prompting (few-shot, chain-of-thought, analogical reasoning) for high-fidelity breakdown assessment. These are integrated into an “escalation” architecture where our efficient detector defers to larger models only when necessary, substantially reducing operational costs and computational overhead. Our fine-tuned model and prompting strategies achieve state-of-the-art performance on DBDC5 and strong results on BETOLD, outperforming specialized classifiers on DBDC5 and narrowing the performance gap to larger proprietary models. The proposed monitor-escalate pipeline reduces inference costs by 54%, providing a cost-effective and interpretable solution for robust conversational AI in high-impact domains. Code and models will be publicly released.

Method: The paper categorizes memory into two forms (parametric/implicit in model weights and contextual/explicit external data), defines six core memory operations (Consolidation, Updating, Indexing, Forgetting, Retrieval, Condensation), and maps these to four key research topics.

Result: Creates a structured taxonomy that reveals four key research areas: long-term memory, long-context memory, parametric modification, and multi-source memory. The framework clarifies functional interactions in LLM-based agents and provides benchmarks and tools for future research.

Conclusion: The taxonomy provides a systematic view of memory-related research, offering guidance for future advancements in LLM-based agents by clarifying the fundamental operations and research directions in memory systems.

Abstract: Memory is fundamental to large language model (LLM)-based agents, but existing surveys emphasize application-level use (e.g., personalized dialogue), while overlooking the atomic operations governing memory dynamics. This work categorizes memory into parametric (implicit in model weights) and contextual (explicit external data, structured/unstructured) forms, and defines six core operations: Consolidation, Updating, Indexing, Forgetting, Retrieval, and Condensation. Mapping these dimensions reveals four key research topics: long-term, long-context, parametric modification, and multi-source memory. The taxonomy provides a structured view of memory-related research, benchmarks, and tools, clarifying functional interactions in LLM-based agents and guiding future advancements. The datasets, papers, and tools are publicly available at https://github.com/Elvin-Yiming-Du/Survey_Memory_in_AI.

Method: Proposes Prune-on-Logic, a structure-aware framework that transforms Long-CoT into logic graphs and selectively prunes low-utility reasoning steps under self-verification constraints. Analyzes three pruning strategies: entire chains, core reasoning, and verification.

Result: Verification pruning consistently improves accuracy while reducing token usage, whereas pruning reasoning steps or indiscriminate pruning degrades performance. Gains hold across tasks, model scales, and CoT capability, with larger models benefiting more from pruning.

Conclusion: Effective pruning aligns supervision with model capacity rather than merely shortening inputs. Pruning serves as a structural optimization strategy for aligning CoT reasoning with SLM capacity, revealing that richer but more redundant reasoning in larger models benefits more from pruning.

Abstract: Long chain-of-thought (Long-CoT) reasoning improves accuracy in LLMs, yet its verbose, self-reflective style often hinders effective distillation into small language models (SLMs). We revisit Long-CoT compression through the lens of capability alignment and ask: Can pruning improve reasoning? We propose Prune-on-Logic, a structure-aware framework that transforms Long-CoT into logic graphs and selectively prunes low-utility reasoning steps under self-verification constraints. Through systematic analysis across three pruning strategies targeting entire chains, core reasoning, and verification, we find that verification pruning consistently improves accuracy while reducing token usage, whereas pruning reasoning steps or indiscriminate pruning degrades performance. Our study reveals that effective pruning aligns supervision with model capacity rather than merely shortening inputs. Gains hold across tasks, model scales, and CoT capability, with larger models benefiting more from pruning due to richer but more redundant reasoning. Our empirical findings highlight pruning as a structural optimization strategy for aligning CoT reasoning with SLM capacity.

Method: Systematic evaluation of Qwen3 (1.7B-235B) and DeepSeek-R1 (1.5B-70B) models across 15 medical datasets with controlled thinking budgets ranging from zero to unlimited tokens, using both native API and truncation methods.

Result: Identified logarithmic scaling relationships, three efficiency regimes (high-efficiency: 0-256 tokens, balanced: 256-512 tokens, high-accuracy: >512 tokens), and found smaller models benefit disproportionately more (15-20% improvements) from extended thinking than larger models (5-10%). Domain-specific patterns show neurology/gastroenterology require deeper reasoning than cardiovascular/respiratory medicine.

Conclusion: Thinking budget control is critical for optimizing medical AI systems, enabling dynamic resource allocation aligned with clinical needs while maintaining transparency essential for healthcare deployment, with validated generalizability across architectures.

Abstract: This study presents the first comprehensive evaluation of thinking budget mechanisms in medical reasoning tasks, revealing fundamental scaling laws between computational resources and reasoning quality. We systematically evaluated two major model families, Qwen3 (1.7B to 235B parameters) and DeepSeek-R1 (1.5B to 70B parameters), across 15 medical datasets spanning diverse specialties and difficulty levels. Through controlled experiments with thinking budgets ranging from zero to unlimited tokens, we establish logarithmic scaling relationships where accuracy improvements follow a predictable pattern with both thinking budget and model size. Our findings identify three distinct efficiency regimes: high-efficiency (0 to 256 tokens) suitable for real-time applications, balanced (256 to 512 tokens) offering optimal cost-performance tradeoffs for routine clinical support, and high-accuracy (above 512 tokens) justified only for critical diagnostic tasks. Notably, smaller models demonstrate disproportionately larger benefits from extended thinking, with 15 to 20% improvements compared to 5 to 10% for larger models, suggesting a complementary relationship where thinking budget provides greater relative benefits for capacity-constrained models. Domain-specific patterns emerge clearly, with neurology and gastroenterology requiring significantly deeper reasoning processes than cardiovascular or respiratory medicine. The consistency between Qwen3 native thinking budget API and our proposed truncation method for DeepSeek-R1 validates the generalizability of thinking budget concepts across architectures. These results establish thinking budget control as a critical mechanism for optimizing medical AI systems, enabling dynamic resource allocation aligned with clinical needs while maintaining the transparency essential for healthcare deployment.

Method: Developed GAICo as a unified, extensible framework with high-level API for end-to-end analysis, supporting reference-based metrics for unstructured text, structured data, and multimedia, plus visualization and reporting capabilities.

Result: Successfully deployed library used in multi-modal AI Travel Assistant case study; downloaded over 13K times on PyPI within 2 months, showing strong community adoption.

Conclusion: GAICo enables reproducible GenAI evaluation, accelerates development, and helps build more trustworthy AI systems by providing standardized comparison tools across diverse output types.

Abstract: The rapid proliferation of Generative AI (GenAI) into diverse, high-stakes domains necessitates robust and reproducible evaluation methods. However, practitioners often resort to ad-hoc, non-standardized scripts, as common metrics are often unsuitable for specialized, structured outputs (e.g., automated plans, time-series) or holistic comparison across modalities (e.g., text, audio, and image). This fragmentation hinders comparability and slows AI system development. To address this challenge, we present GAICo (Generative AI Comparator): a deployed, open-source Python library that streamlines and standardizes GenAI output comparison. GAICo provides a unified, extensible framework supporting a comprehensive suite of reference-based metrics for unstructured text, specialized structured data formats, and multimedia (images, audio). Its architecture features a high-level API for rapid, end-to-end analysis, from multi-model comparison to visualization and reporting, alongside direct metric access for granular control. We demonstrate GAICo’s utility through a detailed case study evaluating and debugging complex, multi-modal AI Travel Assistant pipelines. GAICo empowers AI researchers and developers to efficiently assess system performance, make evaluation reproducible, improve development velocity, and ultimately build more trustworthy AI systems, aligning with the goal of moving faster and safer in AI deployment. Since its release on PyPI in Jun 2025, the tool has been downloaded over 13K times, across versions, by Aug 2025, demonstrating growing community interest.

Method: Proposes LLM2Comp using context compression as pretext task: model learns to generate compact memory tokens that substitute the whole context for downstream sequence prediction. Combines with contrastive learning for further improvements.

Result: Compression objective significantly enhances LLM-based text representations, outperforming token-level pretext tasks. LLM2Comp beats contemporary LLM-based encoders on wide range of tasks while being more sample-efficient, requiring less training data.

Conclusion: Context compression is an effective pretext task for unsupervised LLM adaptation, producing superior text representations with better sample efficiency than existing token-level methods.

Abstract: Text representation plays a critical role in tasks like clustering, retrieval, and other downstream applications. With the emergence of large language models (LLMs), there is increasing interest in harnessing their capabilities for this purpose. However, most of the LLMs are inherently causal and optimized for next-token prediction, making them suboptimal for producing holistic representations. To address this, recent studies introduced pretext tasks to adapt LLMs for text representation. Most of these tasks, however, rely on token-level prediction objectives, such as the masked next-token prediction (MNTP) used in LLM2Vec. In this work, we explore the untapped potential of context compression as a pretext task for unsupervised adaptation of LLMs. During compression pre-training, the model learns to generate compact memory tokens, which substitute the whole context for downstream sequence prediction. Experiments demonstrate that a well-designed compression objective can significantly enhance LLM-based text representations, outperforming models trained with token-level pretext tasks. Further improvements through contrastive learning produce a strong representation model (LLM2Comp) that outperforms contemporary LLM-based text encoders on a wide range of tasks while being more sample-efficient, requiring significantly less training data. Code is available at https://github.com/longtaizi13579/LLM2Comp.

Method: Created a robust evaluation framework with 2,800 carefully curated Chinese medical exam questions across 7 specialties (cardiovascular, gastroenterology, hematology, infectious diseases, nephrology, neurology, respiratory medicine) at two professional levels (attending physician and senior physician). Evaluated 27 state-of-the-art LLMs using this benchmark.

Result: Mixtral-8x7B achieved highest overall accuracy (74.25%), followed by DeepSeek-R1-671B (64.07%). No consistent correlation between model size and performance. Significant performance gaps across specialties - better on cardiovascular/neurology vs gastroenterology/nephrology. Top models showed minimal performance degradation between difficulty levels.

Conclusion: The benchmark provides critical insights for LLM deployment in medical education and clinical decision support, highlighting both promise and current limitations. Smaller mixture-of-experts architectures can outperform larger models, and robust generalization capabilities exist for top-performing models across difficulty levels.

Abstract: The rapid advancement of large language models(LLMs) has prompted significant interest in their potential applications in medical domains. This paper presents a comprehensive benchmark evaluation of 27 state-of-the-art LLMs on Chinese medical examination questions, encompassing seven medical specialties across two professional levels. We introduce a robust evaluation framework that assesses model performance on 2,800 carefully curated questions from cardiovascular, gastroenterology, hematology, infectious diseases, nephrology, neurology, and respiratory medicine domains. Our dataset distinguishes between attending physician and senior physician difficulty levels, providing nuanced insights into model capabilities across varying complexity. Our empirical analysis reveals substantial performance variations among models, with Mixtral-8x7B achieving the highest overall accuracy of 74.25%, followed by DeepSeek-R1-671B at 64.07%. Notably, we observe no consistent correlation between model size and performance, as evidenced by the strong performance of smaller mixture-of-experts architectures. The evaluation demonstrates significant performance gaps between medical specialties, with models generally performing better on cardiovascular and neurology questions compared to gastroenterology and nephrology domains. Furthermore, our analysis indicates minimal performance degradation between attending and senior physician levels for top-performing models, suggesting robust generalization capabilities. This benchmark provides critical insights for the deployment of LLMs in medical education and clinical decision support systems, highlighting both the promise and current limitations of these technologies in specialized medical contexts.

Method: Tournament-style ELO ranking with multi-agent reasoning where multiple LLM agents compete, critique, and collaborate through structured workflows with configurable parameters, dynamic agent selection, and consensus fusion strategies.

Result: Significant improvements in response accuracy, coherence, and reliability with 8.4% overall quality improvement and R^2 values exceeding 0.96 in ELO rating convergence.

Conclusion: ART provides a scalable, production-ready solution for high-quality, vetted LLM responses through systematic optimization of outputs via tournament-based multi-agent collaboration.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, single-model responses often exhibit inconsistencies, hallucinations, and varying quality across different query domains. This paper presents ART (Adaptive Response Tuning), a novel framework that employs tournament-style ELO ranking and multi-agent reasoning to systematically optimize LLM outputs. By enabling multiple LLM agents to compete, critique, and collaborate through structured tournament workflows, ART produces consensus responses that outperform individual model outputs. Our framework introduces configurable tournament parameters, dynamic agent selection, and multiple consensus fusion strategies. Experimental evaluations demonstrate significant improvements in response accuracy, coherence, and reliability compared to baseline single-model approaches. The ART framework provides a scalable, production-ready solution for applications requiring high-quality, vetted LLM responses, achieving an 8.4% improvement in overall quality metrics and R^2 values exceeding 0.96 in ELO rating convergence.

Method: Created VLegal-Bench using Bloom’s cognitive taxonomy to design tasks reflecting practical legal usage scenarios. Developed a rigorous annotation pipeline where legal experts label and cross-validate 10,450 samples, ensuring grounding in authoritative legal documents and mirroring real-world legal assistant workflows.

Result: Established the first comprehensive benchmark for Vietnamese legal tasks, providing a standardized, transparent, and cognitively informed evaluation framework. The benchmark includes diverse tasks like general legal Q&A, retrieval-augmented generation, multi-step reasoning, and scenario-based problem solving.

Conclusion: VLegal-Bench provides a solid foundation for assessing LLM performance in Vietnamese legal contexts and supports development of more reliable, interpretable, and ethically aligned AI-assisted legal systems. The benchmark is publicly available to facilitate access and reproducibility.

Abstract: The rapid advancement of large language models (LLMs) has enabled new possibilities for applying artificial intelligence within the legal domain. Nonetheless, the complexity, hierarchical organization, and frequent revisions of Vietnamese legislation pose considerable challenges for evaluating how well these models interpret and utilize legal knowledge. To address this gap, the Vietnamese Legal Benchmark (VLegal-Bench) is introduced, the first comprehensive benchmark designed to systematically assess LLMs on Vietnamese legal tasks. Informed by Bloom’s cognitive taxonomy, VLegal-Bench encompasses multiple levels of legal understanding through tasks designed to reflect practical usage scenarios. The benchmark comprises 10,450 samples generated through a rigorous annotation pipeline, where legal experts label and cross-validate each instance using our annotation system to ensure every sample is grounded in authoritative legal documents and mirrors real-world legal assistant workflows, including general legal questions and answers, retrieval-augmented generation, multi-step reasoning, and scenario-based problem solving tailored to Vietnamese law. By providing a standardized, transparent, and cognitively informed evaluation framework, VLegal-Bench establishes a solid foundation for assessing LLM performance in Vietnamese legal contexts and supports the development of more reliable, interpretable, and ethically aligned AI-assisted legal systems. To facilitate access and reproducibility, we provide a public landing page for this benchmark at https://vilegalbench.cmcai.vn/.

Method: Adapts pretrained Gemma 3 decoder-only models into encoder-decoder architecture using UL2 adaptation recipe, extends to multimodal capabilities, and introduces two efficiency improvements: tied word embeddings (shared across encoder/decoder) and merged attention (unifying decoder self- and cross-attention).

Result: Shows generality of adaptation strategy across architectures and modalities, demonstrates encoder-decoder strength in long-context modeling, achieves comparable/better pretraining performance and significantly improved post-training performance compared to Gemma 3 counterparts. Releases three model sizes (270M-270M, 1B-1B, 4B-4B) to community.

Conclusion: T5Gemma 2 successfully extends the adaptation approach to multimodal settings while improving efficiency, validating the encoder-decoder architecture’s advantages for long-context tasks and providing open models for future research.

Abstract: We introduce T5Gemma 2, the next generation of the T5Gemma family of lightweight open encoder-decoder models, featuring strong multilingual, multimodal and long-context capabilities. T5Gemma 2 follows the adaptation recipe (via UL2) in T5Gemma – adapting a pretrained decoder-only model into an encoder-decoder model, and extends it from text-only regime to multimodal based on the Gemma 3 models. We further propose two methods to improve the efficiency: tied word embedding that shares all embeddings across encoder and decoder, and merged attention that unifies decoder self- and cross-attention into a single joint module. Experiments demonstrate the generality of the adaptation strategy over architectures and modalities as well as the unique strength of the encoder-decoder architecture on long context modeling. Similar to T5Gemma, T5Gemma 2 yields comparable or better pretraining performance and significantly improved post-training performance than its Gemma 3 counterpart. We release the pretrained models (270M-270M, 1B-1B and 4B-4B) to the community for future research.

Method: The paper introduces an evaluation framework that reports: 1) boundary density, 2) segment alignment diagnostics (purity and coverage), and 3) window-tolerant F1 (W-F1). This separates boundary scoring from boundary selection, allowing evaluation across different density regimes rather than at a single operating point.

Result: Cross-dataset evaluation shows that reported performance differences often reflect annotation granularity mismatch rather than boundary placement quality alone. Boundary-based metrics are strongly coupled to boundary density, with threshold sweeps producing larger W-F1 changes than switching between methods.

Conclusion: Topic segmentation should be viewed as a granularity selection problem rather than prediction of a single correct boundary set. This motivates separating boundary scoring from boundary selection for analyzing and tuning segmentation under varying annotation granularities.

Abstract: Dialogue topic segmentation supports summarization, retrieval, memory management, and conversational continuity. Despite decades of work, evaluation practice remains dominated by strict boundary matching and F1-based metrics. Modern large language model (LLM) based conversational systems increasingly rely on segmentation to manage conversation history beyond fixed context windows. In such systems, unstructured context accumulation degrades efficiency and coherence. This paper introduces an evaluation framework that reports boundary density and segment alignment diagnostics (purity and coverage) alongside window-tolerant F1 (W-F1). By separating boundary scoring from boundary selection, we evaluate segmentation quality across density regimes rather than at a single operating point. Cross-dataset evaluation shows that reported performance differences often reflect annotation granularity mismatch rather than boundary placement quality alone. We evaluate structurally distinct segmentation strategies across eight dialogue datasets spanning task-oriented, open-domain, meeting-style, and synthetic interactions. Boundary-based metrics are strongly coupled to boundary density: threshold sweeps produce larger W-F1 changes than switching between methods. These findings support viewing topic segmentation as a granularity selection problem rather than prediction of a single correct boundary set. This motivates separating boundary scoring from boundary selection for analyzing and tuning segmentation under varying annotation granularities.

Method: Proposes a human-centered framework organized around three dimensions: augmented terminologist (AI as capability amplifier), ethical AI, and human-centered design, building on AI and translation studies research.

Result: A framework emphasizing compatibility of high automation with strong human control, terminologists’ central role in bias mitigation, and designing AI tools around terminologists’ needs, values, and well-being.

Conclusion: Current AI adoption choices will shape not only terminological practice but also the preservation of accuracy, adequacy, and diversity in terminology and specialized knowledge.

Abstract: The rapid diffusion of generative artificial intelligence is transforming terminology work. While this technology promises gains in efficiency, its unstructured adoption risks weakening professional autonomy, amplifying bias, and eroding linguistic and conceptual diversity. This paper argues that a human-centered approach to artificial intelligence has become a necessity for terminology work. Building on research in artificial intelligence and translation studies, it proposes a human-centered framework that conceptualizes artificial intelligence as a means of amplifying the terminologist’s capabilities, rather than replacing them. The framework is organized around three interrelated dimensions: the augmented terminologist, ethical AI, and human-centered design. Together, these dimensions emphasize the compatibility of high automation with strong human control, the central role of terminologists in bias mitigation, and the importance of designing AI tools and workflows around the needs, values, and well-being of the terminologist. The paper concludes by stressing that current choices in AI adoption will shape not only terminological practice, but also the preservation of accuracy, adequacy, and diversity in terminology and specialized knowledge.

Method: Proposes M³KG-RAG with two key components: 1) A lightweight multi-agent pipeline to construct multi-hop multimodal knowledge graphs (M³KG) with context-enriched triplets of multimodal entities, enabling modality-wise retrieval. 2) GRASP (Grounded Retrieval And Selective Pruning) for precise entity grounding, answer-supporting relevance evaluation, and pruning redundant context.

Result: Extensive experiments across diverse multimodal benchmarks demonstrate that M³KG-RAG significantly enhances MLLMs’ multimodal reasoning and grounding capabilities compared to existing approaches.

Conclusion: M³KG-RAG effectively addresses limitations in multimodal RAG by improving retrieval precision through multi-hop knowledge graphs and selective pruning, leading to better reasoning depth and answer faithfulness in audio-visual domains.

Abstract: Retrieval-Augmented Generation (RAG) has recently been extended to multimodal settings, connecting multimodal large language models (MLLMs) with vast corpora of external knowledge such as multimodal knowledge graphs (MMKGs). Despite their recent success, multimodal RAG in the audio-visual domain remains challenging due to 1) limited modality coverage and multi-hop connectivity of existing MMKGs, and 2) retrieval based solely on similarity in a shared multimodal embedding space, which fails to filter out off-topic or redundant knowledge. To address these limitations, we propose M$^3$KG-RAG, a Multi-hop Multimodal Knowledge Graph-enhanced RAG that retrieves query-aligned audio-visual knowledge from MMKGs, improving reasoning depth and answer faithfulness in MLLMs. Specifically, we devise a lightweight multi-agent pipeline to construct multi-hop MMKG (M$^3$KG), which contains context-enriched triplets of multimodal entities, enabling modality-wise retrieval based on input queries. Furthermore, we introduce GRASP (Grounded Retrieval And Selective Pruning), which ensures precise entity grounding to the query, evaluates answer-supporting relevance, and prunes redundant context to retain only knowledge essential for response generation. Extensive experiments across diverse multimodal benchmarks demonstrate that M$^3$KG-RAG significantly enhances MLLMs’ multimodal reasoning and grounding over existing approaches.

Method: 1) Data Synthesis Strategy Based on Atomic Capabilities to reinforce planning and report writing; 2) Progressive training path from agentic mid-training to SFT and RL; 3) Checklist-style Judger for improved robustness; 4) Established ADR-Bench for Chinese domain evaluation.

Result: Step-DeepResearch (32B) scores 61.4% on Scale AI Research Rubrics. On ADR-Bench, it significantly outperforms comparable models and rivals SOTA closed-source models like OpenAI and Gemini DeepResearch.

Conclusion: Refined training enables medium-sized models to achieve expert-level capabilities at industry-leading cost-efficiency, bridging the gap between academic benchmarks and real-world deep research demands.

Abstract: As LLMs shift toward autonomous agents, Deep Research has emerged as a pivotal metric. However, existing academic benchmarks like BrowseComp often fail to meet real-world demands for open-ended research, which requires robust skills in intent recognition, long-horizon decision-making, and cross-source verification. To address this, we introduce Step-DeepResearch, a cost-effective, end-to-end agent. We propose a Data Synthesis Strategy Based on Atomic Capabilities to reinforce planning and report writing, combined with a progressive training path from agentic mid-training to SFT and RL. Enhanced by a Checklist-style Judger, this approach significantly improves robustness. Furthermore, to bridge the evaluation gap in the Chinese domain, we establish ADR-Bench for realistic deep research scenarios. Experimental results show that Step-DeepResearch (32B) scores 61.4% on Scale AI Research Rubrics. On ADR-Bench, it significantly outperforms comparable models and rivals SOTA closed-source models like OpenAI and Gemini DeepResearch. These findings prove that refined training enables medium-sized models to achieve expert-level capabilities at industry-leading cost-efficiency.

Method: NeRV360 integrates viewport extraction directly into the decoding process and introduces a spatial-temporal affine transform module for conditional decoding based on viewpoint and time. This allows decoding only the user-selected viewport rather than reconstructing the entire panoramic frame.

Result: On 6K-resolution videos, NeRV360 achieves a 7-fold reduction in memory consumption and a 2.5-fold increase in decoding speed compared to HNeRV (a representative prior work), while delivering better image quality in terms of objective metrics.

Conclusion: NeRV360 successfully addresses the memory and speed limitations of previous NeRV approaches for 360° videos by integrating viewport extraction into decoding, enabling practical real-time applications for high-resolution 360° video compression.

Abstract: Implicit neural representations for videos (NeRV) have shown strong potential for video compression. However, applying NeRV to high-resolution 360-degree videos causes high memory usage and slow decoding, making real-time applications impractical. We propose NeRV360, an end-to-end framework that decodes only the user-selected viewport instead of reconstructing the entire panoramic frame. Unlike conventional pipelines, NeRV360 integrates viewport extraction into decoding and introduces a spatial-temporal affine transform module for conditional decoding based on viewpoint and time. Experiments on 6K-resolution videos show that NeRV360 achieves a 7-fold reduction in memory consumption and a 2.5-fold increase in decoding speed compared to HNeRV, a representative prior work, while delivering better image quality in terms of objective metrics.

Method: Introduced VL4Gaze benchmark with 489K automatically generated question-answer pairs across 124K images, formulating gaze understanding as a unified VQA problem through four tasks: gaze object description, gaze direction description, gaze point location, and ambiguous question recognition.

Result: Large-scale VLMs struggle to reliably infer gaze semantics and spatial localization without task-specific supervision. Training on VL4Gaze brings substantial and consistent improvements across all tasks, highlighting the importance of targeted multi-task supervision.

Conclusion: Gaze understanding does not naturally emerge from general-purpose vision-language pre-training and requires targeted multi-task supervision. The VL4Gaze benchmark enables systematic evaluation and development of gaze understanding capabilities in VLMs.

Abstract: Human gaze provides essential cues for interpreting attention, intention, and social interaction in visual scenes, yet gaze understanding remains largely unexplored in current vision-language models (VLMs). While recent VLMs achieve strong scene-level reasoning across a range of visual tasks, there exists no benchmark that systematically evaluates or trains them for gaze interpretation, leaving open the question of whether gaze understanding can emerge from general-purpose vision-language pre-training. To address this gap, we introduce VL4Gaze, the first large-scale benchmark designed to investigate, evaluate, and unlock the potential of VLMs for gaze understanding. VL4Gaze contains 489K automatically generated question-answer pairs across 124K images and formulates gaze understanding as a unified VQA problem through four complementary tasks: (1) gaze object description, (2) gaze direction description, (3) gaze point location, and (4) ambiguous question recognition. We comprehensively evaluate both commercial and open-source VLMs under in-context learning and fine-tuning settings. The results show that even large-scale VLMs struggle to reliably infer gaze semantics and spatial localization without task-specific supervision. In contrast, training on VL4Gaze brings substantial and consistent improvements across all tasks, highlighting the importance of targeted multi-task supervision for developing gaze understanding capabilities in VLMs. We will release the dataset and code to support further research and development in this direction.

Method: Iterative hardware-aware neural architecture search using Once-for-All-style ResDets supernet with evolutionary search alternating between backbone and neck/head optimization, plus population passthrough and accuracy predictor.

Result: TrashDet-l achieves 19.5 mAP50 with 30.5M parameters (3.6 mAP50 improvement over prior detectors). Family spans 1.2M-30.5M parameters with 11.4-19.5 mAP50. On MAX78002 microcontroller, variants reduce energy by 88%, latency by 78%, power by 53% vs baselines.

Conclusion: The proposed framework successfully creates scalable, deployment-ready trash detectors for TinyML devices, offering significant improvements in accuracy, efficiency, and resource utilization for diverse hardware constraints.

Abstract: This paper addresses trash detection on the TACO dataset under strict TinyML constraints using an iterative hardware-aware neural architecture search framework targeting edge and IoT devices. The proposed method constructs a Once-for-All-style ResDets supernet and performs iterative evolutionary search that alternates between backbone and neck/head optimization, supported by a population passthrough mechanism and an accuracy predictor to reduce search cost and improve stability. This framework yields a family of deployment-ready detectors, termed TrashDets. On a five-class TACO subset (paper, plastic, bottle, can, cigarette), the strongest variant, TrashDet-l, achieves 19.5 mAP50 with 30.5M parameters, improving accuracy by up to 3.6 mAP50 over prior detectors while using substantially fewer parameters. The TrashDet family spans 1.2M to 30.5M parameters with mAP50 values between 11.4 and 19.5, providing scalable detector options for diverse TinyML deployment budgets on resource-constrained hardware. On the MAX78002 microcontroller with the TrashNet dataset, two specialized variants, TrashDet-ResNet and TrashDet-MBNet, jointly dominate the ai87-fpndetector baseline, with TrashDet-ResNet achieving 7525~$μ$J energy per inference at 26.7 ms latency and 37.45 FPS, and TrashDet-MBNet improving mAP50 by 10.2%; together they reduce energy consumption by up to 88%, latency by up to 78%, and average power by up to 53% compared to existing TinyML detectors.

Method: Introduces OccuFly benchmark captured at 50m, 40m, and 30m altitudes across four seasons. Proposes LiDAR-free data generation framework using camera modality with traditional 3D reconstruction. Automates label transfer by lifting annotated 2D masks into reconstructed point clouds to minimize manual 3D annotation effort.

Result: Created comprehensive benchmark covering urban, industrial, and rural scenarios with 22 semantic classes. Data format adheres to established conventions for integration with existing research. Benchmarked state-of-the-art methods and highlighted challenges specific to elevated viewpoints.

Conclusion: OccuFly provides the first real-world camera-based aerial SSC benchmark, enabling holistic 3D scene understanding from UAV perspectives. The LiDAR-free framework reduces annotation effort and addresses practical constraints of aerial robotics, advancing research in aerial 3D perception.

Abstract: Semantic Scene Completion (SSC) is crucial for 3D perception in mobile robotics, as it enables holistic scene understanding by jointly estimating dense volumetric occupancy and per-voxel semantics. Although SSC has been widely studied in terrestrial domains such as autonomous driving, aerial scenarios like autonomous flying remain largely unexplored, thereby limiting progress on downstream applications. Furthermore, LiDAR sensors represent the primary modality for SSC data generation, which poses challenges for most uncrewed aerial vehicles (UAVs) due to flight regulations, mass and energy constraints, and the sparsity of LiDAR-based point clouds from elevated viewpoints. To address these limitations, we introduce OccuFly, the first real-world, camera-based aerial SSC benchmark, captured at altitudes of 50m, 40m, and 30m during spring, summer, fall, and winter. OccuFly covers urban, industrial, and rural scenarios, provides 22 semantic classes, and the data format adheres to established conventions to facilitate seamless integration with existing research. Crucially, we propose a LiDAR-free data generation framework based on camera modality, which is ubiquitous on modern UAVs. By utilizing traditional 3D reconstruction, our framework automates label transfer by lifting a subset of annotated 2D masks into the reconstructed point cloud, thereby substantially minimizing manual 3D annotation effort. Finally, we benchmark the state-of-the-art on OccuFly and highlight challenges specific to elevated viewpoints, yielding a comprehensive vision benchmark for holistic aerial 3D scene understanding.

Method: Proposes NullBUS framework with nullable prompts implemented as learnable null embeddings with presence masks, enabling fallback to image-only evidence when metadata is absent while still using text when available.

Result: Achieves mean IoU of 0.8568 and mean Dice of 0.9103 on unified pool of three public BUS datasets, demonstrating state-of-the-art performance under mixed prompt availability.

Conclusion: NullBUS effectively addresses the challenge of missing metadata in public BUS datasets through a mixed-supervision approach that handles both prompt-present and prompt-absent scenarios in a single model.

Abstract: Breast ultrasound (BUS) segmentation provides lesion boundaries essential for computer-aided diagnosis and treatment planning. While promptable methods can improve segmentation performance and tumor delineation when text or spatial prompts are available, many public BUS datasets lack reliable metadata or reports, constraining training to small multimodal subsets and reducing robustness. We propose NullBUS, a multimodal mixed-supervision framework that learns from images with and without prompts in a single model. To handle missing text, we introduce nullable prompts, implemented as learnable null embeddings with presence masks, enabling fallback to image-only evidence when metadata are absent and the use of text when present. Evaluated on a unified pool of three public BUS datasets, NullBUS achieves a mean IoU of 0.8568 and a mean Dice of 0.9103, demonstrating state-of-the-art performance under mixed prompt availability.

Method: Task-driven co-design framework unifying optics, sensor modeling, and semantic segmentation networks into single end-to-end RAW-to-task pipeline. Includes realistic cellphone-scale lens models, learnable color filter arrays, Poisson-Gaussian noise processes, and quantization, all optimized directly for segmentation objectives.

Result: Consistent mIoU improvements over fixed pipelines on KITTI-360, with optics modeling and CFA learning providing largest gains, especially for thin or low-light-sensitive classes. Achieved with compact ~1M-parameter model running at ~28 FPS, demonstrating edge deployability.

Conclusion: Full-stack co-optimization of optics, sensors, and networks is a principled path toward efficient, reliable, and deployable perception in autonomous systems, with co-designed sensors adapting acquisition to semantic structure and maintaining accuracy under challenging conditions.

Abstract: Traditional autonomous driving pipelines decouple camera design from downstream perception, relying on fixed optics and handcrafted ISPs that prioritize human viewable imagery rather than machine semantics. This separation discards information during demosaicing, denoising, or quantization, while forcing models to adapt to sensor artifacts. We present a task-driven co-design framework that unifies optics, sensor modeling, and lightweight semantic segmentation networks into a single end-to-end RAW-to-task pipeline. Building on DeepLens[19], our system integrates realistic cellphone-scale lens models, learnable color filter arrays, Poisson-Gaussian noise processes, and quantization, all optimized directly for segmentation objectives. Evaluations on KITTI-360 show consistent mIoU improvements over fixed pipelines, with optics modeling and CFA learning providing the largest gains, especially for thin or low-light-sensitive classes. Importantly, these robustness gains are achieved with a compact ~1M-parameter model running at ~28 FPS, demonstrating edge deployability. Visual and quantitative analyses further highlight how co-designed sensors adapt acquisition to semantic structure, sharpening boundaries and maintaining accuracy under blur, noise, and low bit-depth. Together, these findings establish full-stack co-optimization of optics, sensors, and networks as a principled path toward efficient, reliable, and deployable perception in autonomous systems.

Method: Created CHAMMI-75 dataset by curating heterogeneous, multi-channel microscopy images from 75 diverse biological studies from publicly available sources to train channel-adaptive models.

Result: Training with CHAMMI-75 improves performance in multi-channel bioimaging tasks due to its high diversity in microscopy modalities, enabling models to process any microscopy image type.

Conclusion: CHAMMI-75 paves the way for next-generation cellular morphology models that are channel-adaptive and can be reused across diverse biological studies.

Abstract: Quantifying cell morphology using images and machine learning has proven to be a powerful tool to study the response of cells to treatments. However, models used to quantify cellular morphology are typically trained with a single microscopy imaging type. This results in specialized models that cannot be reused across biological studies because the technical specifications do not match (e.g., different number of channels), or because the target experimental conditions are out of distribution. Here, we present CHAMMI-75, an open access dataset of heterogeneous, multi-channel microscopy images from 75 diverse biological studies. We curated this resource from publicly available sources to investigate cellular morphology models that are channel-adaptive and can process any microscopy image type. Our experiments show that training with CHAMMI-75 can improve performance in multi-channel bioimaging tasks primarily because of its high diversity in microscopy modalities. This work paves the way to create the next generation of cellular morphology models for biological studies.

Method: TVP first solves problems with basic tools while accumulating solutions in an Example Library, then abstracts recurring patterns into reusable higher-level tools for an evolving Tool Library.

Result: Achieves SOTA on Omni3D-Bench (outperforming GPT-4o by 22% and previous best by 11%), with transductive tools used 5x more frequently and showing strong generalization to unseen spatial tasks.

Conclusion: Experience-driven transductive tool creation is a powerful paradigm for building self-evolving visual programming agents that effectively tackle challenging spatial reasoning tasks.

Abstract: Spatial reasoning in 3D scenes requires precise geometric calculations that challenge vision-language models. Visual programming addresses this by decomposing problems into steps calling specialized tools, yet existing methods rely on either fixed toolsets or speculative tool induction before solving problems, resulting in suboptimal programs and poor utilization of induced tools. We present Transductive Visual Programming (TVP), a novel framework that builds new tools from its own experience rather than speculation. TVP first solves problems using basic tools while accumulating experiential solutions into an Example Library, then abstracts recurring patterns from these programs into reusable higher-level tools for an evolving Tool Library. This allows TVP to tackle new problems with increasingly powerful tools learned from experience. On Omni3D-Bench, TVP achieves state-of-the-art performance, outperforming GPT-4o by 22% and the previous best visual programming system by 11%. Our transductively learned tools are used 5x more frequently as core program dependency than inductively created ones, demonstrating more effective tool discovery and reuse. The evolved tools also show strong generalization to unseen spatial tasks, achieving superior performance on benchmarks from SpatialScore-Hard collection without any testset-specific modification. Our work establishes experience-driven transductive tool creation as a powerful paradigm for building self-evolving visual programming agents that effectively tackle challenging spatial reasoning tasks. We release our code at https://transductive-visualprogram.github.io/.

Method: Leverages TinyML techniques, wireless communication framework, and microcontroller platforms to develop a multimodal sensing system that fuses accelerometer data and vision inputs for image classification, object detection, and behavior recognition tasks.

Result: Achieves up to 270× model size reduction, less than 80ms response latency, on-par performance with existing methods, and demonstrates seamless wireless data transmission for remote deployment.

Conclusion: Delivers a robust, scalable IoT-edge livestock monitoring solution adaptable to diverse farming needs with flexibility for future extensions, enabling efficient real-time animal monitoring in resource-constrained environments.

Abstract: The advancement of technology has revolutionized the agricultural industry, transitioning it from labor-intensive farming practices to automated, AI-powered management systems. In recent years, more intelligent livestock monitoring solutions have been proposed to enhance farming efficiency and productivity. This work presents a novel approach to animal activity recognition and movement tracking, leveraging tiny machine learning (TinyML) techniques, wireless communication framework, and microcontroller platforms to develop an efficient, cost-effective livestock sensing system. It collects and fuses accelerometer data and vision inputs to build a multimodal network for three tasks: image classification, object detection, and behavior recognition. The system is deployed and evaluated on commercial microcontrollers for real-time inference using embedded applications, demonstrating up to 270$\times$ model size reduction, less than 80ms response latency, and on-par performance comparable to existing methods. The incorporation of the wireless communication technique allows for seamless data transmission between devices, benefiting use cases in remote locations with poor Internet connectivity. This work delivers a robust, scalable IoT-edge livestock monitoring solution adaptable to diverse farming needs, offering flexibility for future extensions.

Method: Proposes adaptive visual preprocessing combining content-aware image analysis, adaptive resolution selection, and content-aware cropping to reduce visual redundancy before vision encoding. Integrated with FastVLM without modifying architecture or requiring retraining.

Result: On DocVQA subset: reduces per-image inference time by over 50%, lowers mean full generation time, and achieves >55% reduction in visual token count compared to baseline pipeline.

Conclusion: Input-aware preprocessing is an effective and lightweight strategy for improving deployment-oriented efficiency of vision-language models, demonstrating significant computational savings without architectural changes.

Abstract: Vision-Language Models (VLMs) have demonstrated strong performance on multimodal reasoning tasks, but their deployment remains challenging due to high inference latency and computational cost, particularly when processing high-resolution visual inputs. While recent architectures such as FastVLM improve efficiency through optimized vision encoders, existing pipelines still rely on static visual preprocessing, leading to redundant computation for visually simple inputs. In this work, we propose an adaptive visual preprocessing method that dynamically adjusts input resolution and spatial coverage based on image content characteristics. The proposed approach combines content-aware image analysis, adaptive resolution selection, and content-aware cropping to reduce visual redundancy prior to vision encoding. Importantly, the method is integrated with FastVLM without modifying its architecture or requiring retraining. We evaluate the proposed method on a subset of the DocVQA dataset in an inference-only setting, focusing on efficiency-oriented metrics. Experimental results show that adaptive preprocessing reduces per-image inference time by over 50%, lowers mean full generation time, and achieves a consistent reduction of more than 55% in visual token count compared to the baseline pipeline. These findings demonstrate that input-aware preprocessing is an effective and lightweight strategy for improving deployment-oriented efficiency of vision-language models. To facilitate reproducibility, our implementation is provided as a fork of the FastVLM repository, incorporating the files for the proposed method, and is available at https://github.com/kmdavidds/mlfastlm.

Method: ALIVE operates fully on local hardware with three key components: (1) Avatar-delivered lectures using ASR transcription, LLM refinement, and neural talking-head synthesis; (2) Content-aware retrieval combining semantic similarity with timestamp alignment; (3) Real-time multimodal interaction allowing pause, questions via text/voice, and avatar/text responses.

Result: Demonstrated on a complete medical imaging course, ALIVE shows accurate retrieval, good latency characteristics, and positive user experience. The system provides accurate, content-aware, and engaging real-time support.

Conclusion: ALIVE shows how multimodal AI combined with content-aware retrieval and local deployment can significantly enhance recorded lectures’ pedagogical value, offering an extensible pathway toward next-generation interactive learning environments.

Abstract: Traditional lecture videos offer flexibility but lack mechanisms for real-time clarification, forcing learners to search externally when confusion arises. Recent advances in large language models and neural avatars provide new opportunities for interactive learning, yet existing systems typically lack lecture awareness, rely on cloud-based services, or fail to integrate retrieval and avatar-delivered explanations in a unified, privacy-preserving pipeline. We present ALIVE, an Avatar-Lecture Interactive Video Engine that transforms passive lecture viewing into a dynamic, real-time learning experience. ALIVE operates fully on local hardware and integrates (1) Avatar-delivered lecture generated through ASR transcription, LLM refinement, and neural talking-head synthesis; (2) A content-aware retrieval mechanism that combines semantic similarity with timestamp alignment to surface contextually relevant lecture segments; and (3) Real-time multimodal interaction, enabling students to pause the lecture, ask questions through text or voice, and receive grounded explanations either as text or as avatar-delivered responses. To maintain responsiveness, ALIVE employs lightweight embedding models, FAISS-based retrieval, and segmented avatar synthesis with progressive preloading. We demonstrate the system on a complete medical imaging course, evaluate its retrieval accuracy, latency characteristics, and user experience, and show that ALIVE provides accurate, content-aware, and engaging real-time support. ALIVE illustrates how multimodal AI-when combined with content-aware retrieval and local deployment-can significantly enhance the pedagogical value of recorded lectures, offering an extensible pathway toward next-generation interactive learning environments.

Method: 1) B/C/D-Scan three-view joint analysis with FDTD simulation validation; 2) DCO-YOLO framework integrating DySample, CGLU, and OutlookAttention into YOLOv11; 3) 3D-DIoU spatial feature matching algorithm with geometric constraints.

Result: Achieves 96.2% accuracy, 93.3% recall, and 96.7% mAP in complex multi-pipeline scenarios, outperforming baseline by 2.0%, 2.1%, and 0.9% respectively. Ablation studies validate module effectiveness.

Conclusion: Integrates deep learning optimization with 3D GPR physical characteristics, providing efficient and reliable technical framework for intelligent underground pipeline recognition and localization.

Abstract: To address the issues of weak correlation between multi-view features, low recognition accuracy of small-scale targets, and insufficient robustness in complex scenarios in underground pipeline detection using 3D GPR, this paper proposes a 3D pipeline intelligent detection framework. First, based on a B/C/D-Scan three-view joint analysis strategy, a three-dimensional pipeline three-view feature evaluation method is established by cross-validating forward simulation results obtained using FDTD methods with actual measurement data. Second, the DCO-YOLO framework is proposed, which integrates DySample, CGLU, and OutlookAttention cross-dimensional correlation mechanisms into the original YOLOv11 algorithm, significantly improving the small-scale pipeline edge feature extraction capability. Furthermore, a 3D-DIoU spatial feature matching algorithm is proposed, which integrates three-dimensional geometric constraints and center distance penalty terms to achieve automated association of multi-view annotations. The three-view fusion strategy resolves inherent ambiguities in single-view detection. Experiments based on real urban underground pipeline data show that the proposed method achieves accuracy, recall, and mean average precision of 96.2%, 93.3%, and 96.7%, respectively, in complex multi-pipeline scenarios, which are 2.0%, 2.1%, and 0.9% higher than the baseline model. Ablation experiments validated the synergistic optimization effect of the dynamic feature enhancement module and Grad-CAM++ heatmap visualization demonstrated that the improved model significantly enhanced its ability to focus on pipeline geometric features. This study integrates deep learning optimization strategies with the physical characteristics of 3D GPR, offering an efficient and reliable novel technical framework for the intelligent recognition and localization of underground pipelines.

Method: DRI keeps VFM frozen, uses lightweight Offset Encoder to extract domain-specific representations, adaptively transforms them via Modulator guided by contextual features, and injects them into intermediate layers via additive fusion without altering pre-trained weights.

Result: Achieves SOTA performance with minimal parameters: 57.9% and 60.5% mAP on HOSS-ReID dataset using only 1.54M and 7.05M parameters respectively.

Conclusion: DRI effectively bridges modality gaps in cross-modality ship re-ID by optimizing in feature space rather than weight space, preserving general knowledge while adapting to downstream tasks with minimal trainable parameters.

Abstract: Cross-Modality Ship Re-Identification (CMS Re-ID) is critical for achieving all-day and all-weather maritime target tracking, yet it is fundamentally challenged by significant modality discrepancies. Mainstream solutions typically rely on explicit modality alignment strategies; however, this paradigm heavily depends on constructing large-scale paired datasets for pre-training. To address this, grounded in the Platonic Representation Hypothesis, we explore the potential of Vision Foundation Models (VFMs) in bridging modality gaps. Recognizing the suboptimal performance of existing generic Parameter-Efficient Fine-Tuning (PEFT) methods that operate within the weight space, particularly on limited-capacity models, we shift the optimization perspective to the feature space and propose a novel PEFT strategy termed Domain Representation Injection (DRI). Specifically, while keeping the VFM fully frozen to maximize the preservation of general knowledge, we design a lightweight, learnable Offset Encoder to extract domain-specific representations rich in modality and identity attributes from raw inputs. Guided by the contextual information of intermediate features at different layers, a Modulator adaptively transforms these representations. Subsequently, they are injected into the intermediate layers via additive fusion, dynamically reshaping the feature distribution to adapt to the downstream task without altering the VFM’s pre-trained weights. Extensive experimental results demonstrate the superiority of our method, achieving State-of-the-Art (SOTA) performance with minimal trainable parameters. For instance, on the HOSS-ReID dataset, we attain 57.9% and 60.5% mAP using only 1.54M and 7.05M parameters, respectively. The code is available at https://github.com/TingfengXian/DRI.

Method: Proposes DGSAN with: 1) Global-Local Feature Encoder to capture local, global, and fused nodule characteristics; 2) Dual-Graph Construction organizing multimodal features into inter-modal and intra-modal graphs; 3) Hierarchical Cross-Modal Graph Fusion Module for refined feature integration. Also introduces NLST-cmst multimodal dataset.

Result: Extensive experiments on NLST-cmst and CSTL-derived datasets show DGSAN significantly outperforms state-of-the-art methods in pulmonary nodule classification with exceptional computational efficiency.

Conclusion: The proposed DGSAN framework effectively addresses limitations of existing multimodal fusion methods and demonstrates superior performance for lung cancer nodule classification using temporal variations and multimodal data.

Abstract: Lung cancer continues to be the leading cause of cancer-related deaths globally. Early detection and diagnosis of pulmonary nodules are essential for improving patient survival rates. Although previous research has integrated multimodal and multi-temporal information, outperforming single modality and single time point, the fusion methods are limited to inefficient vector concatenation and simple mutual attention, highlighting the need for more effective multimodal information fusion. To address these challenges, we introduce a Dual-Graph Spatiotemporal Attention Network, which leverages temporal variations and multimodal data to enhance the accuracy of predictions. Our methodology involves developing a Global-Local Feature Encoder to better capture the local, global, and fused characteristics of pulmonary nodules. Additionally, a Dual-Graph Construction method organizes multimodal features into inter-modal and intra-modal graphs. Furthermore, a Hierarchical Cross-Modal Graph Fusion Module is introduced to refine feature integration. We also compiled a novel multimodal dataset named the NLST-cmst dataset as a comprehensive source of support for related research. Our extensive experiments, conducted on both the NLST-cmst and curated CSTL-derived datasets, demonstrate that our DGSAN significantly outperforms state-of-the-art methods in classifying pulmonary nodules with exceptional computational efficiency.

Method: 1) Created first comprehensive benchmark with over 1M compressed images using various codecs and tasks; 2) Analyzed performance gap sources (information loss vs. generalization failure); 3) Proposed universal VLM adaptor to enhance compressed image understanding.

Result: Benchmark reveals VLM performance gaps on compressed images; visualization shows only generalization gap can be mitigated; single universal adaptor improves VLM performance by 10-30% across different codecs and bitrates.

Conclusion: The benchmark and enhancement method provide valuable insights for bridging the gap between VLMs and compressed images, enabling more efficient VLM deployment with compressed visual inputs.

Abstract: With the rapid development of Vision-Language Models (VLMs) and the growing demand for their applications, efficient compression of the image inputs has become increasingly important. Existing VLMs predominantly digest and understand high-bitrate compressed images, while their ability to interpret low-bitrate compressed images has yet to be explored by far. In this paper, we introduce the first comprehensive benchmark to evaluate the ability of VLM against compressed images, varying existing widely used image codecs and diverse set of tasks, encompassing over one million compressed images in our benchmark. Next, we analyse the source of performance gap, by categorising the gap from a) the information loss during compression and b) generalisation failure of VLM. We visualize these gaps with concrete examples and identify that for compressed images, only the generalization gap can be mitigated. Finally, we propose a universal VLM adaptor to enhance model performance on images compressed by existing codecs. Consequently, we demonstrate that a single adaptor can improve VLM performance across images with varying codecs and bitrates by 10%-30%. We believe that our benchmark and enhancement method provide valuable insights and contribute toward bridging the gap between VLMs and compressed images.

Method: Three-stage pipeline: 1) Place panoramic viewpoints based on scene layout/geometry, 2) Ground text queries on each panoramic rendering using pretrained 2D VLMs (augmented with 3D semantic/geometric features), 3) Fuse per-view predictions into single 3D bounding box via lifting.

Result: Achieves state-of-the-art results on ScanRefer and Nr3D benchmarks. Demonstrates superior generalization to unseen 3D datasets and text rephrasings.

Conclusion: Panoramic renderings with 3D features effectively bridge 2D VLMs and 3D reasoning, enabling strong generalization in 3D visual grounding while leveraging powerful pretrained 2D vision-language models.

Abstract: 3D Visual Grounding (3DVG) is a critical bridge from vision-language perception to robotics, requiring both language understanding and 3D scene reasoning. Traditional supervised models leverage explicit 3D geometry but exhibit limited generalization, owing to the scarcity of 3D vision-language datasets and the limited reasoning capabilities compared to modern vision-language models (VLMs). We propose PanoGrounder, a generalizable 3DVG framework that couples multi-modal panoramic representation with pretrained 2D VLMs for strong vision-language reasoning. Panoramic renderings, augmented with 3D semantic and geometric features, serve as an intermediate representation between 2D and 3D, and offer two major benefits: (i) they can be directly fed to VLMs with minimal adaptation and (ii) they retain long-range object-to-object relations thanks to their 360-degree field of view. We devise a three-stage pipeline that places a compact set of panoramic viewpoints considering the scene layout and geometry, grounds a text query on each panoramic rendering with a VLM, and fuses per-view predictions into a single 3D bounding box via lifting. Our approach achieves state-of-the-art results on ScanRefer and Nr3D, and demonstrates superior generalization to unseen 3D datasets and text rephrasings.

Method: Proposes SMC-Mamba with three key components: 1) Modality-Agnostic Feature Enhancement (MAFE) module for detail preservation and global representation enhancement, 2) Multiplex Consensus Cross-modal Mamba (MCCM) module for dynamic expert collaboration and cross-modal feature integration, and 3) Bi-level Self-supervised Contrastive Learning Loss (BSCL) for high-frequency preservation without extra computation.

Result: Extensive experiments show the approach outperforms state-of-the-art image fusion algorithms in infrared-visible, medical, multi-focus, and multi-exposure fusion tasks, as well as downstream visual tasks.

Conclusion: SMC-Mamba provides an effective self-supervised framework for general image fusion that achieves superior performance across multiple fusion domains and enhances downstream task performance through efficient cross-modal information integration.

Abstract: Image fusion integrates complementary information from different modalities to generate high-quality fused images, thereby enhancing downstream tasks such as object detection and semantic segmentation. Unlike task-specific techniques that primarily focus on consolidating inter-modal information, general image fusion needs to address a wide range of tasks while improving performance without increasing complexity. To achieve this, we propose SMC-Mamba, a Self-supervised Multiplex Consensus Mamba framework for general image fusion. Specifically, the Modality-Agnostic Feature Enhancement (MAFE) module preserves fine details through adaptive gating and enhances global representations via spatial-channel and frequency-rotational scanning. The Multiplex Consensus Cross-modal Mamba (MCCM) module enables dynamic collaboration among experts, reaching a consensus to efficiently integrate complementary information from multiple modalities. The cross-modal scanning within MCCM further strengthens feature interactions across modalities, facilitating seamless integration of critical information from both sources. Additionally, we introduce a Bi-level Self-supervised Contrastive Learning Loss (BSCL), which preserves high-frequency information without increasing computational overhead while simultaneously boosting performance in downstream tasks. Extensive experiments demonstrate that our approach outperforms state-of-the-art (SOTA) image fusion algorithms in tasks such as infrared-visible, medical, multi-focus, and multi-exposure fusion, as well as downstream visual tasks.

Method: Proposes Quantile Rendering (Q-Render) that sparsely samples only dominant 3D Gaussians along each ray instead of conventional dense sampling. Also introduces Gaussian Splatting Network (GS-Net), a generalizable 3D neural network that predicts Gaussian features.

Result: Outperforms state-of-the-art methods on ScanNet and LeRF benchmarks while enabling real-time rendering with approximately ~43.7x speedup on 512-D feature maps compared to existing approaches.

Conclusion: Q-Render provides an efficient solution for high-dimensional feature rendering in 3D open-vocabulary segmentation, maintaining high fidelity while achieving significant speed improvements, making real-time applications feasible.

Abstract: Recent advancements in computer vision have successfully extended Open-vocabulary segmentation (OVS) to the 3D domain by leveraging 3D Gaussian Splatting (3D-GS). Despite this progress, efficiently rendering the high-dimensional features required for open-vocabulary queries poses a significant challenge. Existing methods employ codebooks or feature compression, causing information loss, thereby degrading segmentation quality. To address this limitation, we introduce Quantile Rendering (Q-Render), a novel rendering strategy for 3D Gaussians that efficiently handles high-dimensional features while maintaining high fidelity. Unlike conventional volume rendering, which densely samples all 3D Gaussians intersecting each ray, Q-Render sparsely samples only those with dominant influence along the ray. By integrating Q-Render into a generalizable 3D neural network, we also propose Gaussian Splatting Network (GS-Net), which predicts Gaussian features in a generalizable manner. Extensive experiments on ScanNet and LeRF demonstrate that our framework outperforms state-of-the-art methods, while enabling real-time rendering with an approximate ~43.7x speedup on 512-D feature maps. Code will be made publicly available.

Method: A Collaborative Multi-Agent Reasoning Framework that explicitly separates Semantic Planning from Visual Synthesis. It includes: (1) a self-correcting Verification Agent using Chain-of-Thought reasoning to fix segmentation and identify occluders, and (2) a Diverse Hypothesis Generator for multiple plausible semantic interpretations of invisible regions. Also introduces MAC-Score, a novel MLLM-based evaluation metric.

Result: The framework significantly outperforms state-of-the-art methods across multiple datasets, validated against human judgment and ground truth.

Conclusion: By decoupling reasoning from synthesis and employing specialized agents for upfront planning, the approach enables visually and semantically coherent amodal completion in a single pass, establishing a robust standard for assessing structural completeness and semantic consistency.

Abstract: Amodal completion, the task of inferring invisible object parts, faces significant challenges in maintaining semantic consistency and structural integrity. Prior progressive approaches are inherently limited by inference instability and error accumulation. To tackle these limitations, we present a Collaborative Multi-Agent Reasoning Framework that explicitly decouples Semantic Planning from Visual Synthesis. By employing specialized agents for upfront reasoning, our method generates a structured, explicit plan before pixel generation, enabling visually and semantically coherent single-pass synthesis. We integrate this framework with two critical mechanisms: (1) a self-correcting Verification Agent that employs Chain-of-Thought reasoning to rectify visible region segmentation and identify residual occluders strictly within the Semantic Planning phase, and (2) a Diverse Hypothesis Generator that addresses the ambiguity of invisible regions by offering diverse, plausible semantic interpretations, surpassing the limited pixel-level variations of standard random seed sampling. Furthermore, addressing the limitations of traditional metrics in assessing inferred invisible content, we introduce the MAC-Score (MLLM Amodal Completion Score), a novel human-aligned evaluation metric. Validated against human judgment and ground truth, these metrics establish a robust standard for assessing structural completeness and semantic consistency with visible context. Extensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods across multiple datasets. Our project is available at: https://fanhongxing.github.io/remac-page.

Method: Proposes Real-centric Envelope Modeling (REM) that shifts detection from learning generator artifacts to modeling robust distribution of real images. Introduces feature-level perturbations in self-reconstruction to generate near-real samples, and employs an envelope estimator with cross-domain consistency to learn a boundary enclosing the real image manifold. Also builds RealChain benchmark covering open-source and commercial generators with simulated real-world degradation.

Result: Across eight benchmark evaluations, REM achieves average improvement of 7.5% over state-of-the-art methods. Notably maintains exceptional generalization on severely degraded RealChain benchmark, establishing solid foundation for synthetic image detection under real-world conditions.

Conclusion: REM provides a new paradigm for robust synthetic image detection that focuses on modeling real image distribution rather than generator artifacts, demonstrating superior performance and generalization to real-world conditions with chain degradations.

Abstract: The rapid progress of generative models has intensified the need for reliable and robust detection under real-world conditions. However, existing detectors often overfit to generator-specific artifacts and remain highly sensitive to real-world degradations. As generative architectures evolve and images undergo multi-round cross-platform sharing and post-processing (chain degradations), these artifact cues become obsolete and harder to detect. To address this, we propose Real-centric Envelope Modeling (REM), a new paradigm that shifts detection from learning generator artifacts to modeling the robust distribution of real images. REM introduces feature-level perturbations in self-reconstruction to generate near-real samples, and employs an envelope estimator with cross-domain consistency to learn a boundary enclosing the real image manifold. We further build RealChain, a comprehensive benchmark covering both open-source and commercial generators with simulated real-world degradation. Across eight benchmark evaluations, REM achieves an average improvement of 7.5% over state-of-the-art methods, and notably maintains exceptional generalization on the severely degraded RealChain benchmark, establishing a solid foundation for synthetic image detection under real-world conditions. The code and the RealChain benchmark will be made publicly available upon acceptance of the paper.

Method: SPOT represents road structures and CCTV placement as documents using 2D spatial coordinates with chunking for real-time querying. It transforms vehicle positions to world coordinates using relative position and FOV information, then combines map spatial data with vehicle movement patterns to perform beam search at intersections for predicting next CCTV locations.

Result: Experimental results using CARLA simulator in a virtual city environment show SPOT accurately predicts next CCTV locations in blind spot sections, maintaining continuous vehicle trajectories more effectively than existing techniques.

Conclusion: SPOT successfully addresses blind spot tracking challenges in multi-CCTV environments without prior training, enabling reliable continuous vehicle trajectory prediction through map-guided spatial reasoning and LLM-based inference.

Abstract: CCTV-based vehicle tracking systems face structural limitations in continuously connecting the trajectories of the same vehicle across multiple camera environments. In particular, blind spots occur due to the intervals between CCTVs and limited Fields of View (FOV), which leads to object ID switching and trajectory loss, thereby reducing the reliability of real-time path prediction. This paper proposes SPOT (Spatial Prediction Over Trajectories), a map-guided LLM agent capable of tracking vehicles even in blind spots of multi-CCTV environments without prior training. The proposed method represents road structures (Waypoints) and CCTV placement information as documents based on 2D spatial coordinates and organizes them through chunking techniques to enable real-time querying and inference. Furthermore, it transforms the vehicle’s position into the actual world coordinate system using the relative position and FOV information of objects observed in CCTV images. By combining map spatial information with the vehicle’s moving direction, speed, and driving patterns, a beam search is performed at the intersection level to derive candidate CCTV locations where the vehicle is most likely to enter after the blind spot. Experimental results based on the CARLA simulator in a virtual city environment confirmed that the proposed method accurately predicts the next appearing CCTV even in blind spot sections, maintaining continuous vehicle trajectories more effectively than existing techniques.

Method: Proposes XGrid-Mapping, a hybrid grid framework that combines: 1) sparse explicit grid for geometric priors and structural guidance, 2) implicit dense grid for rich scene representation, 3) VDB structure with submap-based organization for computational efficiency, 4) distillation-based overlap alignment to ensure consistency across submaps, and 5) dynamic removal module for robustness and sampling efficiency.

Result: Extensive experiments show superior mapping quality while overcoming efficiency limitations of voxel-guided methods, outperforming existing state-of-the-art mapping methods in both performance and real-time capability.

Conclusion: XGrid-Mapping successfully addresses the trade-off between geometric structure utilization and computational efficiency in neural LiDAR mapping, enabling efficient large-scale incremental mapping through its hybrid explicit-implicit representation framework with submap organization and consistency alignment mechanisms.

Abstract: Large-scale incremental mapping is fundamental to the development of robust and reliable autonomous systems, as it underpins incremental environmental understanding with sequential inputs for navigation and decision-making. LiDAR is widely used for this purpose due to its accuracy and robustness. Recently, neural LiDAR mapping has shown impressive performance; however, most approaches rely on dense implicit representations and underutilize geometric structure, while existing voxel-guided methods struggle to achieve real-time performance. To address these challenges, we propose XGrid-Mapping, a hybrid grid framework that jointly exploits explicit and implicit representations for efficient neural LiDAR mapping. Specifically, the strategy combines a sparse grid, providing geometric priors and structural guidance, with an implicit dense grid that enriches scene representation. By coupling the VDB structure with a submap-based organization, the framework reduces computational load and enables efficient incremental mapping on a large scale. To mitigate discontinuities across submaps, we introduce a distillation-based overlap alignment strategy, in which preceding submaps supervise subsequent ones to ensure consistency in overlapping regions. To further enhance robustness and sampling efficiency, we incorporate a dynamic removal module. Extensive experiments show that our approach delivers superior mapping quality while overcoming the efficiency limitations of voxel-guided methods, thereby outperforming existing state-of-the-art mapping methods.

Method: Proposed pipeline: 1) Train diffusion model on abundant normal X-rays, 2) Use pre-trained model to inpaint head lesions into diseased X-rays, preserving tail classes as augmented data, 3) Integrate Large Language Model Knowledge Guidance (LKG) module, 4) Apply Progressive Incremental Learning (PIL) strategy to stabilize inpainting fine-tuning.

Result: Comprehensive evaluations on public lung datasets MIMIC and CheXpert demonstrate that the proposed method sets a new benchmark in performance for diagnosing long-tailed pulmonary anomalies.

Conclusion: The proposed data synthesis pipeline effectively addresses the data scarcity problem for rare lung lesions by leveraging abundant normal X-rays and advanced generative techniques, significantly improving diagnostic accuracy for long-tailed pulmonary anomalies in chest radiography.

Abstract: Long-tailed pulmonary anomalies in chest radiography present formidable diagnostic challenges. Despite the recent strides in diffusion-based methods for enhancing the representation of tailed lesions, the paucity of rare lesion exemplars curtails the generative capabilities of these approaches, thereby leaving the diagnostic precision less than optimal. In this paper, we propose a novel data synthesis pipeline designed to augment tail lesions utilizing a copious supply of conventional normal X-rays. Specifically, a sufficient quantity of normal samples is amassed to train a diffusion model capable of generating normal X-ray images. This pre-trained diffusion model is subsequently utilized to inpaint the head lesions present in the diseased X-rays, thereby preserving the tail classes as augmented training data. Additionally, we propose the integration of a Large Language Model Knowledge Guidance (LKG) module alongside a Progressive Incremental Learning (PIL) strategy to stabilize the inpainting fine-tuning process. Comprehensive evaluations conducted on the public lung datasets MIMIC and CheXpert demonstrate that the proposed method sets a new benchmark in performance.

Method: Two-stage flow-matching: first learns direct flow from sparse to dense, then refines with noise-perturbed samples. Includes adaptive time scheduler for efficient sampling, on-manifold constraints to keep points on surface, and recurrent interface network (RIN) for hierarchical feature interactions.

Result: Sets new state-of-the-art on synthetic benchmarks and real-world scans, delivering superior visual fidelity and quantitative accuracy across various tasks. Code and models publicly available.

Conclusion: PUFM++ advances point cloud upsampling through enhanced flow-matching with improved geometric fidelity, robustness, and task consistency, demonstrating strong performance on both synthetic and real-world data.

Abstract: Recent advances in generative modeling have demonstrated strong promise for high-quality point cloud upsampling. In this work, we present PUFM++, an enhanced flow-matching framework for reconstructing dense and accurate point clouds from sparse, noisy, and partial observations. PUFM++ improves flow matching along three key axes: (i) geometric fidelity, (ii) robustness to imperfect input, and (iii) consistency with downstream surface-based tasks. We introduce a two-stage flow-matching strategy that first learns a direct, straight-path flow from sparse inputs to dense targets, and then refines it using noise-perturbed samples to approximate the terminal marginal distribution better. To accelerate and stabilize inference, we propose a data-driven adaptive time scheduler that improves sampling efficiency based on interpolation behavior. We further impose on-manifold constraints during sampling to ensure that generated points remain aligned with the underlying surface. Finally, we incorporate a recurrent interface network~(RIN) to strengthen hierarchical feature interactions and boost reconstruction quality. Extensive experiments on synthetic benchmarks and real-world scans show that PUFM++ sets a new state of the art in point cloud upsampling, delivering superior visual fidelity and quantitative accuracy across a wide range of tasks. Code and pretrained models are publicly available at https://github.com/Holmes-Alan/Enhanced_PUFM.

Method: Feed-forward neural network with alternating attention across views to capture intra-view lighting and inter-view material consistency. Uses consistency-based finetuning with unlabeled real-world videos to improve generalization.

Result: Achieves state-of-the-art performance in multi-view consistency, material/normal estimation quality, and generalization to real-world imagery on benchmark datasets.

Conclusion: The proposed framework enables efficient, consistent multi-view inverse rendering with strong real-world generalization through novel attention mechanisms and consistency-based finetuning.

Abstract: Multi-view inverse rendering aims to recover geometry, materials, and illumination consistently across multiple viewpoints. When applied to multi-view images, existing single-view approaches often ignore cross-view relationships, leading to inconsistent results. In contrast, multi-view optimization methods rely on slow differentiable rendering and per-scene refinement, making them computationally expensive and hard to scale. To address these limitations, we introduce a feed-forward multi-view inverse rendering framework that directly predicts spatially varying albedo, metallic, roughness, diffuse shading, and surface normals from sequences of RGB images. By alternating attention across views, our model captures both intra-view long-range lighting interactions and inter-view material consistency, enabling coherent scene-level reasoning within a single forward pass. Due to the scarcity of real-world training data, models trained on existing synthetic datasets often struggle to generalize to real-world scenes. To overcome this limitation, we propose a consistency-based finetuning strategy that leverages unlabeled real-world videos to enhance both multi-view coherence and robustness under in-the-wild conditions. Extensive experiments on benchmark datasets demonstrate that our method achieves state-of-the-art performance in terms of multi-view consistency, material and normal estimation quality, and generalization to real-world imagery.

Method: NExT-Vid uses masked next-frame prediction to jointly model images and videos. It introduces a context-isolated autoregressive predictor to decouple semantic representation from target decoding, and a conditioned flow-matching decoder to enhance generation quality and diversity.

Result: Extensive experiments on large-scale pretrained models show that NExT-Vid consistently outperforms previous generative pretraining methods for visual representation learning via attentive probing in downstream classification tasks.

Conclusion: NExT-Vid provides an effective autoregressive visual generative pretraining framework that addresses limitations of existing methods and achieves strong visual representations through context-isolated flow-matching pretraining.

Abstract: Recent advances in pretraining general foundation models have significantly improved performance across diverse downstream tasks. While autoregressive (AR) generative models like GPT have revolutionized NLP, most visual generative pretraining methods still rely on BERT-style masked modeling, which often disregards the temporal information essential for video analysis. The few existing autoregressive visual pretraining methods suffer from issues such as inaccurate semantic localization and poor generation quality, leading to poor semantics. In this work, we propose NExT-Vid, a novel autoregressive visual generative pretraining framework that utilizes masked next-frame prediction to jointly model images and videos. NExT-Vid introduces a context-isolated autoregressive predictor to decouple semantic representation from target decoding, and a conditioned flow-matching decoder to enhance generation quality and diversity. Through context-isolated flow-matching pretraining, our approach achieves strong representations. Extensive experiments on large-scale pretrained models demonstrate that our proposed method consistently outperforms previous generative pretraining methods for visual representation learning via attentive probing in downstream classification.

Method: Granular-ball Guided Masking (GBGM) uses Granular-ball Computing (GBC) to guide a coarse-to-fine hierarchical masking process. It adaptively preserves semantically rich, structurally important regions while suppressing redundant areas.

Result: Extensive experiments on multiple benchmarks show consistent improvements in classification accuracy and masked image reconstruction, demonstrating effectiveness and broad applicability.

Conclusion: GBGM provides a simple, model-agnostic structure-aware data augmentation paradigm that integrates seamlessly into CNNs and Vision Transformers, enhancing model robustness without requiring architectural changes.

Abstract: Deep learning models have achieved remarkable success in computer vision, but they still rely heavily on large-scale labeled data and tend to overfit when data are limited or distributions shift. Data augmentation, particularly mask-based information dropping, can enhance robustness by forcing models to explore complementary cues; however, existing approaches often lack structural awareness and may discard essential semantics. We propose Granular-ball Guided Masking (GBGM), a structure-aware augmentation strategy guided by Granular-ball Computing (GBC). GBGM adaptively preserves semantically rich, structurally important regions while suppressing redundant areas through a coarse-to-fine hierarchical masking process, producing augmentations that are both representative and discriminative. Extensive experiments on multiple benchmarks demonstrate consistent improvements in classification accuracy and masked image reconstruction, confirming the effectiveness and broad applicability of the proposed method. Simple and model-agnostic, it integrates seamlessly into CNNs and Vision Transformers and provides a new paradigm for structure-aware data augmentation.

Method: Integrates Mamba (linear time-series module) into pretrained Stable Diffusion models to replace temporal attention layers, uses low-rank approximation matrices for query/key weight matrices in causal attention, and employs weighted averaging during training to update attention scores.

Result: Demonstrates promising results in editing various attributes, subjects, and locations in real-world videos while preserving generative power and reducing computational burden.

Conclusion: FluencyVE provides a simple yet effective approach for one-shot video editing that achieves better temporal consistency with reduced computational costs compared to existing methods.

Abstract: Large-scale text-to-image diffusion models have achieved unprecedented success in image generation and editing. However, extending this success to video editing remains challenging. Recent video editing efforts have adapted pretrained text-to-image models by adding temporal attention mechanisms to handle video tasks. Unfortunately, these methods continue to suffer from temporal inconsistency issues and high computational overheads. In this study, we propose FluencyVE, which is a simple yet effective one-shot video editing approach. FluencyVE integrates the linear time-series module, Mamba, into a video editing model based on pretrained Stable Diffusion models, replacing the temporal attention layer. This enables global frame-level attention while reducing the computational costs. In addition, we employ low-rank approximation matrices to replace the query and key weight matrices in the causal attention, and use a weighted averaging technique during training to update the attention scores. This approach significantly preserves the generative power of the text-to-image model while effectively reducing the computational burden. Experiments and analyses demonstrate promising results in editing various attributes, subjects, and locations in real-world videos.

Method: Proposes an efficient video defense framework that protects portrait videos by perturbing the 3D information acquisition process. Key innovations include: (1) a similarity-guided parameter sharing mechanism for computational efficiency, and (2) a multi-scale dual-domain attention module to jointly optimize spatial-frequency perturbations.

Result: The framework demonstrates strong defense capability, achieves 47x acceleration over the fastest baseline while maintaining high fidelity. It remains robust against scaling operations and state-of-the-art purification attacks, with effectiveness validated through ablation studies.

Conclusion: The proposed video defense framework effectively protects portrait videos against 3D-field talking face generation attacks by efficiently perturbing 3D information acquisition, offering a practical solution to privacy concerns with minimal quality degradation and significant computational efficiency gains.

Abstract: State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from reference portrait video. This capability raises significant privacy concerns regarding malicious misuse of personal portraits. However, no efficient defense framework exists to protect such videos against 3D-field TFG methods. While image-based defenses could apply per-frame 2D perturbations, they incur prohibitive computational costs, severe video quality degradation, failing to disrupt 3D information for video protection. To address this, we propose a novel and efficient video defense framework against 3D-field TFG methods, which protects portrait video by perturbing the 3D information acquisition process while maintain high-fidelity video quality. Specifically, our method introduces: (1) a similarity-guided parameter sharing mechanism for computational efficiency, and (2) a multi-scale dual-domain attention module to jointly optimize spatial-frequency perturbations. Extensive experiments demonstrate that our proposed framework exhibits strong defense capability and achieves a 47x acceleration over the fastest baseline while maintaining high fidelity. Moreover, it remains robust against scaling operations and state-of-the-art purification attacks, and the effectiveness of our design choices is further validated through ablation studies. Our project is available at https://github.com/Richen7418/VDF.

Method: Latent diffusion model for high-quality visible face generation from thermal inputs, multi-attribute classifier for facial attribute extraction, and Self-attn Mamba module for global cross-modal feature modeling and faster inference.

Result: Achieves state-of-the-art performance on two benchmark datasets, with superior image quality and identity preservation compared to existing methods.

Conclusion: The proposed approach effectively addresses domain shift and feature loss challenges in HFR, enabling robust infrared face recognition through high-quality visible image generation with preserved identity features.

Abstract: Modern surveillance systems increasingly rely on multi-wavelength sensors and deep neural networks to recognize faces in infrared images captured at night. However, most facial recognition models are trained on visible light datasets, leading to substantial performance degradation on infrared inputs due to significant domain shifts. Early feature-based methods for infrared face recognition proved ineffective, prompting researchers to adopt generative approaches that convert infrared images into visible light images for improved recognition. This paradigm, known as Heterogeneous Face Recognition (HFR), faces challenges such as model and modality discrepancies, leading to distortion and feature loss in generated images. To address these limitations, this paper introduces a novel latent diffusion-based model designed to generate high-quality visible face images from thermal inputs while preserving critical identity features. A multi-attribute classifier is incorporated to extract key facial attributes from visible images, mitigating feature loss during infrared-to-visible image restoration. Additionally, we propose the Self-attn Mamba module, which enhances global modeling of cross-modal features and significantly improves inference speed. Experimental results on two benchmark datasets demonstrate the superiority of our approach, achieving state-of-the-art performance in both image quality and identity preservation.

Method: Next-Scale Prediction (NSP) constructs cross-scale training pairs where blind-spot networks take low-resolution, fully decorrelated sub-images as input to predict high-resolution targets that retain fine details. This decouples noise decorrelation from detail preservation.

Result: NSP achieves state-of-the-art self-supervised denoising performance on real-world benchmarks, significantly alleviating the conflict between noise decorrelation and detail preservation. As a by-product, it naturally supports super-resolution of noisy images without retraining or modification.

Conclusion: NSP provides an effective solution to the long-standing trade-off in self-supervised denoising by separating noise decorrelation from detail preservation through cross-scale prediction, while offering additional super-resolution capabilities as a valuable by-product.

Abstract: Self-supervised real-world image denoising remains a fundamental challenge, arising from the antagonistic trade-off between decorrelating spatially structured noise and preserving high-frequency details. Existing blind-spot network (BSN) methods rely on pixel-shuffle downsampling (PD) to decorrelate noise, but aggressive downsampling fragments fine structures, while milder downsampling fails to remove correlated noise. To address this, we introduce Next-Scale Prediction (NSP), a novel self-supervised paradigm that decouples noise decorrelation from detail preservation. NSP constructs cross-scale training pairs, where BSN takes low-resolution, fully decorrelated sub-images as input to predict high-resolution targets that retain fine details. As a by-product, NSP naturally supports super-resolution of noisy images without retraining or modification. Extensive experiments demonstrate that NSP achieves state-of-the-art self-supervised denoising performance on real-world benchmarks, significantly alleviating the long-standing conflict between noise decorrelation and detail preservation.

Method: Created KOREATECH-CGH dataset with 6,000 RGB-D image/hologram pairs across multiple resolutions. Introduced amplitude projection technique that replaces amplitude components of hologram wavefields at each depth layer while preserving phase.

Result: Achieved 27.01 dB PSNR and 0.87 SSIM, surpassing previous optimized silhouette-masking layer-based method by 2.03 dB and 0.04 SSIM. Demonstrated utility through hologram generation and super-resolution experiments with state-of-the-art ML models.

Conclusion: KOREATECH-CGH dataset enables training and evaluation of next-generation ML-CGH systems, addressing the data scarcity problem in the field.

Abstract: Machine learning-based computer-generated holography (ML-CGH) has advanced rapidly in recent years, yet progress is constrained by the limited availability of high-quality, large-scale hologram datasets. To address this, we present KOREATECH-CGH, a publicly available dataset comprising 6,000 pairs of RGB-D images and complex holograms across resolutions ranging from 256256 to 20482048, with depth ranges extending to the theoretical limits of the angular spectrum method for wide 3D scene coverage. To improve hologram quality at large depth ranges, we introduce amplitude projection, a post-processing technique that replaces amplitude components of hologram wavefields at each depth layer while preserving phase. This approach enhances reconstruction fidelity, achieving 27.01 dB PSNR and 0.87 SSIM, surpassing a recent optimized silhouette-masking layer-based method by 2.03 dB and 0.04 SSIM, respectively. We further validate the utility of KOREATECH-CGH through experiments on hologram generation and super-resolution using state-of-the-art ML models, confirming its applicability for training and evaluating next-generation ML-CGH systems.

Method: Proposed a reweighted logarithmic norm as a nonconvex surrogate for rank function, solved using alternating direction method of multipliers (ADMM) optimization.

Result: Experimental results on image inpainting show superior performance compared to state-of-the-art LRMC approaches in both visual quality and quantitative metrics.

Conclusion: The reweighted logarithmic norm provides a closer approximation to rank function than existing alternatives, leading to better performance in low-rank matrix completion tasks.

Abstract: Low-rank matrix completion (LRMC) has demonstrated remarkable success in a wide range of applications. To address the NP-hard nature of the rank minimization problem, the nuclear norm is commonly used as a convex and computationally tractable surrogate for the rank function. However, this approach often yields suboptimal solutions due to the excessive shrinkage of singular values. In this letter, we propose a novel reweighted logarithmic norm as a more effective nonconvex surrogate, which provides a closer approximation than many existing alternatives. We efficiently solve the resulting optimization problem by employing the alternating direction method of multipliers (ADMM). Experimental results on image inpainting demonstrate that the proposed method achieves superior performance compared to state-of-the-art LRMC approaches, both in terms of visual quality and quantitative metrics.

Method: 1) 2D-3D hybrid feature extraction detects corners and edges from events and object models. 2) Optical flow of corners is found by maximizing event-associated probability within spatio-temporal windows. 3) Correlation between corners and edges is established using optical flow guidance. 4) 6DoF pose is iteratively optimized by minimizing distances between corners and edges.

Result: Experimental results on both simulated and real events show the method outperforms event-based state-of-the-art methods in accuracy and robustness.

Conclusion: The optical flow-guided approach with event cameras provides an effective solution for 6DoF object pose tracking, overcoming limitations of traditional cameras and demonstrating superior performance compared to existing event-based methods.

Abstract: Object pose tracking is one of the pivotal technologies in multimedia, attracting ever-growing attention in recent years. Existing methods employing traditional cameras encounter numerous challenges such as motion blur, sensor noise, partial occlusion, and changing lighting conditions. The emerging bio-inspired sensors, particularly event cameras, possess advantages such as high dynamic range and low latency, which hold the potential to address the aforementioned challenges. In this work, we present an optical flow-guided 6DoF object pose tracking method with an event camera. A 2D-3D hybrid feature extraction strategy is firstly utilized to detect corners and edges from events and object models, which characterizes object motion precisely. Then, we search for the optical flow of corners by maximizing the event-associated probability within a spatio-temporal window, and establish the correlation between corners and edges guided by optical flow. Furthermore, by minimizing the distances between corners and edges, the 6DoF object pose is iteratively optimized to achieve continuous pose tracking. Experimental results of both simulated and real events demonstrate that our methods outperform event-based state-of-the-art methods in terms of both accuracy and robustness.

Method: DexAvatar uses a novel framework guided by learned 3D hand and body priors to reconstruct bio-mechanically accurate fine-grained hand articulations and body movements from in-the-wild monocular sign language videos.

Result: Achieves 35.11% improvement in body and hand pose estimation compared to state-of-the-art on the SGNify motion capture dataset, the only benchmark available for this task.

Conclusion: DexAvatar provides a significant advancement in 3D human pose reconstruction from sign language videos, addressing critical limitations of existing methods and enabling better data for sign language generation systems.

Abstract: The trend in sign language generation is centered around data-driven generative methods that require vast amounts of precise 2D and 3D human pose data to achieve an acceptable generation quality. However, currently, most sign language datasets are video-based and limited to automatically reconstructed 2D human poses (i.e., keypoints) and lack accurate 3D information. Furthermore, existing state-of-the-art for automatic 3D human pose estimation from sign language videos is prone to self-occlusion, noise, and motion blur effects, resulting in poor reconstruction quality. In response to this, we introduce DexAvatar, a novel framework to reconstruct bio-mechanically accurate fine-grained hand articulations and body movements from in-the-wild monocular sign language videos, guided by learned 3D hand and body priors. DexAvatar achieves strong performance in the SGNify motion capture dataset, the only benchmark available for this task, reaching an improvement of 35.11% in the estimation of body and hand poses compared to the state-of-the-art. The official website of this work is: https://github.com/kaustesseract/DexAvatar.

Method: UniPath implements Multi-Stream Control: 1) Raw-Text stream, 2) High-Level Semantics stream using learnable queries to a frozen pathology MLLM to distill paraphrase-robust Diagnostic Semantic Tokens and expand prompts, and 3) Prototype stream for component-level morphological control via a prototype bank. The authors also curated a 2.65M image-text corpus and 68K finely annotated subset.

Result: UniPath achieves state-of-the-art performance with Patho-FID of 80.9 (51% better than second-best) and fine-grained semantic control achieving 98.7% of real-image quality. The framework demonstrates superior controllable generation capabilities in pathology.

Conclusion: UniPath successfully bridges the gap between understanding and generation in computational pathology by leveraging mature diagnostic understanding for controllable image generation. The framework addresses key challenges through multi-stream control and curated datasets, enabling fine-grained semantic control previously unavailable in pathology image generation.

Abstract: In computational pathology, understanding and generation have evolved along disparate paths: advanced understanding models already exhibit diagnostic-level competence, whereas generative models largely simulate pixels. Progress remains hindered by three coupled factors: the scarcity of large, high-quality image-text corpora; the lack of precise, fine-grained semantic control, which forces reliance on non-semantic cues; and terminological heterogeneity, where diverse phrasings for the same diagnostic concept impede reliable text conditioning. We introduce UniPath, a semantics-driven pathology image generation framework that leverages mature diagnostic understanding to enable controllable generation. UniPath implements Multi-Stream Control: a Raw-Text stream; a High-Level Semantics stream that uses learnable queries to a frozen pathology MLLM to distill paraphrase-robust Diagnostic Semantic Tokens and to expand prompts into diagnosis-aware attribute bundles; and a Prototype stream that affords component-level morphological control via a prototype bank. On the data front, we curate a 2.65M image-text corpus and a finely annotated, high-quality 68K subset to alleviate data scarcity. For a comprehensive assessment, we establish a four-tier evaluation hierarchy tailored to pathology. Extensive experiments demonstrate UniPath’s SOTA performance, including a Patho-FID of 80.9 (51% better than the second-best) and fine-grained semantic control achieving 98.7% of the real-image. The meticulously curated datasets, complete source code, and pre-trained model weights developed in this study will be made openly accessible to the public.

Method: Proposes a self-supervised framework with two strategies: 1) Decomposition - meticulously decomposes fused multimodal features into distinct unimodal features and aligns them with ground truth unimodal counterparts; 2) Composition - integrates multiple unimodal features and uses them as self-supervised guidance to enhance multimodal representation learning.

Result: Extensive experiments on NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD II datasets demonstrate that the method achieves an excellent balance between computational cost and model performance.

Conclusion: The proposed Decomposition and Composition framework successfully addresses the efficiency-effectiveness trade-off in multimodal skeleton-based action recognition through self-supervised learning strategies.

Abstract: Multimodal human action understanding is a significant problem in computer vision, with the central challenge being the effective utilization of the complementarity among diverse modalities while maintaining model efficiency. However, most existing methods rely on simple late fusion to enhance performance, which results in substantial computational overhead. Although early fusion with a shared backbone for all modalities is efficient, it struggles to achieve excellent performance. To address the dilemma of balancing efficiency and effectiveness, we introduce a self-supervised multimodal skeleton-based action representation learning framework, named Decomposition and Composition. The Decomposition strategy meticulously decomposes the fused multimodal features into distinct unimodal features, subsequently aligning them with their respective ground truth unimodal counterparts. On the other hand, the Composition strategy integrates multiple unimodal features, leveraging them as self-supervised guidance to enhance the learning of multimodal representations. Extensive experiments on the NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD II datasets demonstrate that the proposed method strikes an excellent balance between computational cost and model performance.

Method: Uses VGGT backbone for multi-view 3D representations, adapts with feature aggregators, fine-tunes for place recognition. Creates descriptor from 3D tokens and intermediate 2D tokens with dedicated aggregation modules for each. Incorporates single- and multi-frame aggregation schemes with variable-length sequence retrieval.

Result: Sets new state of the art, outperforming both single- and multi-view baselines, demonstrating effectiveness of geometry-grounded tokens for VPR.

Conclusion: UniPR-3D effectively integrates multi-view information for VPR, showing the value of geometry-grounded tokens and multi-view aggregation strategies for improved generalization and performance.

Abstract: Visual Place Recognition (VPR) has been traditionally formulated as a single-image retrieval task. Using multiple views offers clear advantages, yet this setting remains relatively underexplored and existing methods often struggle to generalize across diverse environments. In this work we introduce UniPR-3D, the first VPR architecture that effectively integrates information from multiple views. UniPR-3D builds on a VGGT backbone capable of encoding multi-view 3D representations, which we adapt by designing feature aggregators and fine-tune for the place recognition task. To construct our descriptor, we jointly leverage the 3D tokens and intermediate 2D tokens produced by VGGT. Based on their distinct characteristics, we design dedicated aggregation modules for 2D and 3D features, allowing our descriptor to capture fine-grained texture cues while also reasoning across viewpoints. To further enhance generalization, we incorporate both single- and multi-frame aggregation schemes, along with a variable-length sequence retrieval strategy. Our experiments show that UniPR-3D sets a new state of the art, outperforming both single- and multi-view baselines and highlighting the effectiveness of geometry-grounded tokens for VPR. Our code and models will be made publicly available on Github https://github.com/dtc111111/UniPR-3D.

Method: Proposes a novel multi-task model using non-causal attention to capture entire table structure, combined with a parallel inference algorithm for faster cell content recognition.

Result: Demonstrates superiority both visually and statistically on two large public datasets, showing improved performance and faster inference.

Conclusion: The proposed model effectively addresses table recognition challenges by capturing complete table structure with non-causal attention and achieving faster inference through parallel processing, outperforming existing methods on benchmark datasets.

Abstract: The extraction and use of diverse knowledge from numerous documents is a pressing challenge in intelligent information retrieval. Documents contain elements that require different recognition methods. Table recognition typically consists of three subtasks, namely table structure, cell position and cell content recognition. Recent models have achieved excellent recognition with a combination of multi-task learning, local attention, and mutual learning. However, their effectiveness has not been fully explained, and they require a long period of time for inference. This paper presents a novel multi-task model that utilizes non-causal attention to capture the entire table structure, and a parallel inference algorithm for faster cell content inference. The superiority is demonstrated both visually and statistically on two large public datasets.

Method: Developed T2AV-Compass with 500 diverse prompts via taxonomy-driven pipeline, plus a dual-level evaluation framework integrating objective signal-level metrics (video/audio quality, cross-modal alignment) and subjective MLLM-as-a-Judge protocol.

Result: Evaluation of 11 representative T2AV systems shows even strongest models fall substantially short of human-level realism and cross-modal consistency, with persistent failures in audio realism, fine-grained synchronization, and instruction following.

Conclusion: T2AV-Compass serves as a challenging diagnostic testbed highlighting significant improvement room for future models and advancing text-to-audio-video generation.

Abstract: Text-to-Audio-Video (T2AV) generation aims to synthesize temporally coherent video and semantically synchronized audio from natural language, yet its evaluation remains fragmented, often relying on unimodal metrics or narrowly scoped benchmarks that fail to capture cross-modal alignment, instruction following, and perceptual realism under complex prompts. To address this limitation, we present T2AV-Compass, a unified benchmark for comprehensive evaluation of T2AV systems, consisting of 500 diverse and complex prompts constructed via a taxonomy-driven pipeline to ensure semantic richness and physical plausibility. Besides, T2AV-Compass introduces a dual-level evaluation framework that integrates objective signal-level metrics for video quality, audio quality, and cross-modal alignment with a subjective MLLM-as-a-Judge protocol for instruction following and realism assessment. Extensive evaluation of 11 representative T2AVsystems reveals that even the strongest models fall substantially short of human-level realism and cross-modal consistency, with persistent failures in audio realism, fine-grained synchronization, instruction following, etc. These results indicate significant improvement room for future models and highlight the value of T2AV-Compass as a challenging and diagnostic testbed for advancing text-to-audio-video generation.

Method: 1) Created UniRec40M dataset with 40M text, formula, and mixed samples; 2) Introduced hierarchical supervision training to handle structural variability across document hierarchies; 3) Developed semantic-decoupled tokenizer to separate text and formula representations; 4) Built comprehensive evaluation benchmark covering Chinese/English documents across domains and levels.

Result: UniRec-0.1B outperforms both general-purpose VLMs and leading document parsing expert models while achieving 2-9× speedup. The model demonstrates effectiveness across multiple document levels (characters, words, lines, paragraphs, documents) and languages (Chinese/English).

Conclusion: The proposed lightweight unified recognition model successfully addresses computational efficiency limitations while maintaining strong performance, making unified text and formula recognition more practical for real-world applications through hierarchical supervision and semantic-decoupled tokenization.

Abstract: Text and formulas constitute the core informational components of many documents. Accurately and efficiently recognizing both is crucial for developing robust and generalizable document parsing systems. Recently, vision-language models (VLMs) have achieved impressive unified recognition of text and formulas. However, they are large-sized and computationally demanding, restricting their usage in many applications. In this paper, we propose UniRec-0.1B, a unified recognition model with only 0.1B parameters. It is capable of performing text and formula recognition at multiple levels, including characters, words, lines, paragraphs, and documents. To implement this task, we first establish UniRec40M, a large-scale dataset comprises 40 million text, formula and their mix samples, enabling the training of a powerful yet lightweight model. Secondly, we identify two challenges when building such a lightweight but unified expert model. They are: structural variability across hierarchies and semantic entanglement between textual and formulaic content. To tackle these, we introduce a hierarchical supervision training that explicitly guides structural comprehension, and a semantic-decoupled tokenizer that separates text and formula representations. Finally, we develop a comprehensive evaluation benchmark covering Chinese and English documents from multiple domains and with multiple levels. Experimental results on this and public benchmarks demonstrate that UniRec-0.1B outperforms both general-purpose VLMs and leading document parsing expert models, while achieving a 2-9$\times$ speedup, validating its effectiveness and efficiency. Codebase and Dataset: https://github.com/Topdu/OpenOCR.

Method: FreeInpaint introduces two key techniques: 1) Prior-guided noise optimization that steers model attention toward valid inpainting regions by optimizing initial noise, and 2) A composite guidance objective that directs the denoising process by optimizing intermediate latents at each step to enhance both prompt alignment and visual rationality.

Result: Extensive experiments with various inpainting diffusion models and evaluation metrics demonstrate the effectiveness and robustness of FreeInpaint in improving text-guided image inpainting performance.

Conclusion: FreeInpaint provides a practical tuning-free solution that directly optimizes diffusion latents during inference to achieve better prompt alignment and visual rationality in text-guided image inpainting tasks.

Abstract: Text-guided image inpainting endeavors to generate new content within specified regions of images using textual prompts from users. The primary challenge is to accurately align the inpainted areas with the user-provided prompts while maintaining a high degree of visual fidelity. While existing inpainting methods have produced visually convincing results by leveraging the pre-trained text-to-image diffusion models, they still struggle to uphold both prompt alignment and visual rationality simultaneously. In this work, we introduce FreeInpaint, a plug-and-play tuning-free approach that directly optimizes the diffusion latents on the fly during inference to improve the faithfulness of the generated images. Technically, we introduce a prior-guided noise optimization method that steers model attention towards valid inpainting regions by optimizing the initial noise. Furthermore, we meticulously design a composite guidance objective tailored specifically for the inpainting task. This objective efficiently directs the denoising process, enhancing prompt alignment and visual rationality by optimizing intermediate latents at each step. Through extensive experiments involving various inpainting diffusion models and evaluation metrics, we demonstrate the effectiveness and robustness of our proposed FreeInpaint.

Method: Constructed MarineEval, the first large-scale marine VLM dataset with 2,000 image-based QA pairs covering 7 task dimensions and 20 capacity dimensions. Domain requirements were integrated into data construction and verified by marine experts. Benchmarked 17 existing VLMs on this dataset.

Result: Experimental results show existing VLMs cannot effectively answer domain-specific marine questions, revealing significant performance gaps and substantial room for improvement.

Conclusion: Current VLMs lack marine domain expertise despite general capabilities. MarineEval provides a comprehensive benchmark to evaluate and improve VLM performance in specialized domains, facilitating future research in domain-specific vision-language understanding.

Abstract: We have witnessed promising progress led by large language models (LLMs) and further vision language models (VLMs) in handling various queries as a general-purpose assistant. VLMs, as a bridge to connect the visual world and language corpus, receive both visual content and various text-only user instructions to generate corresponding responses. Though great success has been achieved by VLMs in various fields, in this work, we ask whether the existing VLMs can act as domain experts, accurately answering marine questions, which require significant domain expertise and address special domain challenges/requirements. To comprehensively evaluate the effectiveness and explore the boundary of existing VLMs, we construct the first large-scale marine VLM dataset and benchmark called MarineEval, with 2,000 image-based question-answering pairs. During our dataset construction, we ensure the diversity and coverage of the constructed data: 7 task dimensions and 20 capacity dimensions. The domain requirements are specially integrated into the data construction and further verified by the corresponding marine domain experts. We comprehensively benchmark 17 existing VLMs on our MarineEval and also investigate the limitations of existing models in answering marine research questions. The experimental results reveal that existing VLMs cannot effectively answer the domain-specific questions, and there is still a large room for further performance improvements. We hope our new benchmark and observations will facilitate future research. Project Page: http://marineeval.hkustvgd.com/

Method: TGC-Net introduces three components: 1) Semantic-Structural Synergy Encoder (SSE) that augments CLIP’s ViT with a CNN branch for multi-scale structural refinement, 2) Domain-Augmented Text Encoder (DATE) that injects large-language-model-derived medical knowledge, and 3) Vision-Language Calibration Module (VLCM) that refines cross-modal correspondence in a unified feature space.

Result: Experiments on five datasets across chest X-ray and thoracic CT modalities demonstrate state-of-the-art performance with substantially fewer trainable parameters, including notable Dice gains on challenging benchmarks.

Conclusion: TGC-Net effectively addresses CLIP’s limitations for medical segmentation through parameter-efficient, task-specific adaptations, providing a superior solution for text-guided medical segmentation that balances performance and efficiency.

Abstract: Text-guided medical segmentation enhances segmentation accuracy by utilizing clinical reports as auxiliary information. However, existing methods typically rely on unaligned image and text encoders, which necessitate complex interaction modules for multimodal fusion. While CLIP provides a pre-aligned multimodal feature space, its direct application to medical imaging is limited by three main issues: insufficient preservation of fine-grained anatomical structures, inadequate modeling of complex clinical descriptions, and domain-specific semantic misalignment. To tackle these challenges, we propose TGC-Net, a CLIP-based framework focusing on parameter-efficient, task-specific adaptations. Specifically, it incorporates a Semantic-Structural Synergy Encoder (SSE) that augments CLIP’s ViT with a CNN branch for multi-scale structural refinement, a Domain-Augmented Text Encoder (DATE) that injects large-language-model-derived medical knowledge, and a Vision-Language Calibration Module (VLCM) that refines cross-modal correspondence in a unified feature space. Experiments on five datasets across chest X-ray and thoracic CT modalities demonstrate that TGC-Net achieves state-of-the-art performance with substantially fewer trainable parameters, including notable Dice gains on challenging benchmarks.

Method: Created ORCA benchmark with fine-grained visual/textual annotations capturing morphology-oriented attributes across diverse marine species. Evaluated 18 state-of-the-art models on three tasks: object detection (closed-set and open-vocabulary), instance captioning, and visual grounding.

Result: Results reveal key challenges including species diversity, morphological overlap, and specialized domain demands, highlighting the difficulty of marine understanding despite modern vision models.

Conclusion: ORCA establishes a comprehensive benchmark to advance marine visual understanding research by providing structured evaluation framework and highlighting domain-specific challenges.

Abstract: Marine visual understanding is essential for monitoring and protecting marine ecosystems, enabling automatic and scalable biological surveys. However, progress is hindered by limited training data and the lack of a systematic task formulation that aligns domain-specific marine challenges with well-defined computer vision tasks, thereby limiting effective model application. To address this gap, we present ORCA, a multi-modal benchmark for marine research comprising 14,647 images from 478 species, with 42,217 bounding box annotations and 22,321 expert-verified instance captions. The dataset provides fine-grained visual and textual annotations that capture morphology-oriented attributes across diverse marine species. To catalyze methodological advances, we evaluate 18 state-of-the-art models on three tasks: object detection (closed-set and open-vocabulary), instance captioning, and visual grounding. Results highlight key challenges, including species diversity, morphological overlap, and specialized domain demands, underscoring the difficulty of marine understanding. ORCA thus establishes a comprehensive benchmark to advance research in marine domain. Project Page: http://orca.hkustvgd.com/.

Method: Proposes Equivariant Feature Rotation (EFR) that aligns source and target domains at two complementary levels within a self-rotated proxy feature space. Uses adaptive rotations within a parameterized Lie Group to transform both source and target features into an equivariant proxy space where alignment occurs. Learnable rotation matrices bridge domain gap while preserving intra-domain structural information without distortion.

Result: Comprehensive experiments on various commonly used datasets demonstrate that the method significantly enhances generative performance within the targeted domain compared to existing approaches.

Conclusion: EFR effectively addresses the limitations of traditional consistency constraints in few-shot image generation by introducing a novel alignment strategy in an equivariant proxy feature space, enabling better knowledge transfer from source to target domain while preserving structural information.

Abstract: Few-shot image generation aims to effectively adapt a source generative model to a target domain using very few training images. Most existing approaches introduce consistency constraints-typically through instance-level or distribution-level loss functions-to directly align the distribution patterns of source and target domains within their respective latent spaces. However, these strategies often fall short: overly strict constraints can amplify the negative effects of the domain gap, leading to distorted or uninformative content, while overly relaxed constraints may fail to leverage the source domain effectively. This limitation primarily stems from the inherent discrepancy in the underlying distribution structures of the source and target domains. The scarcity of target samples further compounds this issue by hindering accurate estimation of the target domain’s distribution. To overcome these limitations, we propose Equivariant Feature Rotation (EFR), a novel adaptation strategy that aligns source and target domains at two complementary levels within a self-rotated proxy feature space. Specifically, we perform adaptive rotations within a parameterized Lie Group to transform both source and target features into an equivariant proxy space, where alignment is conducted. These learnable rotation matrices serve to bridge the domain gap by preserving intra-domain structural information without distortion, while the alignment optimization facilitates effective knowledge transfer from the source to the target domain. Comprehensive experiments on a variety of commonly used datasets demonstrate that our method significantly enhances the generative performance within the targeted domain.

Method: Uses Implicit Neural Representations (INRs) with hierarchical temporal encoding to extract multi-scale temporal features, and integrates a custom parametric activation function based on Fourier transformations in the MLP decoder.

Result: Extensive evaluations across benchmark datasets demonstrate the effectiveness and robustness of the approach for representing complex motion behaviors with high accuracy.

Conclusion: The proposed NAME framework successfully creates continuous representations of human motion sequences, enabling flexible temporal manipulation (interpolation, inbetweening, extrapolation) at arbitrary frame rates.

Abstract: Physical motions are inherently continuous, and higher camera frame rates typically contribute to improved smoothness and temporal coherence. For the first time, we explore continuous representations of human motion sequences, featuring the ability to interpolate, inbetween, and even extrapolate any input motion sequences at arbitrary frame rates. To achieve this, we propose a novel parametric activation-induced hierarchical implicit representation framework, referred to as NAME, based on Implicit Neural Representations (INRs). Our method introduces a hierarchical temporal encoding mechanism that extracts features from motion sequences at multiple temporal scales, enabling effective capture of intricate temporal patterns. Additionally, we integrate a custom parametric activation function, powered by Fourier transformations, into the MLP-based decoder to enhance the expressiveness of the continuous representation. This parametric formulation significantly augments the model’s ability to represent complex motion behaviors with high accuracy. Extensive evaluations across several benchmark datasets demonstrate the effectiveness and robustness of our proposed approach.

Method: Two-stage pipeline: 1) Coarse global structure synthesis, 2) Voxel-based refinement with spatial localization decoupled from detail synthesis using RoPE-encoded positional anchors. Includes comprehensive data processing with watertight processing and quality filtering.

Result: Competitive performance with existing open-source methods in both data processing quality and geometry generation, achieving strong geometric quality despite limited training resources.

Conclusion: UltraShape 1.0 provides an effective framework for high-fidelity 3D generation with scalable architecture and improved data processing, with code and models to be released for research.

Abstract: In this report, we introduce UltraShape 1.0, a scalable 3D diffusion framework for high-fidelity 3D geometry generation. The proposed approach adopts a two-stage generation pipeline: a coarse global structure is first synthesized and then refined to produce detailed, high-quality geometry. To support reliable 3D generation, we develop a comprehensive data processing pipeline that includes a novel watertight processing method and high-quality data filtering. This pipeline improves the geometric quality of publicly available 3D datasets by removing low-quality samples, filling holes, and thickening thin structures, while preserving fine-grained geometric details. To enable fine-grained geometry refinement, we decouple spatial localization from geometric detail synthesis in the diffusion process. We achieve this by performing voxel-based refinement at fixed spatial locations, where voxel queries derived from coarse geometry provide explicit positional anchors encoded via RoPE, allowing the diffusion model to focus on synthesizing local geometric details within a reduced, structured solution space. Our model is trained exclusively on publicly available 3D datasets, achieving strong geometric quality despite limited training resources. Extensive evaluations demonstrate that UltraShape 1.0 performs competitively with existing open-source methods in both data processing quality and geometry generation. All code and trained models will be released to support future research.

Method: Created VisRes Bench with 19,000+ controlled task images across three complexity levels: Level 1 tests perceptual completion and image matching under perturbations; Level 2 tests rule-based inference on single attributes; Level 3 tests compositional reasoning integrating multiple attributes.

Result: State-of-the-art VLMs perform near random under subtle perceptual perturbations, revealing limited abstraction beyond pattern recognition and clear limitations in perceptual and relational visual reasoning capacities.

Conclusion: VisRes provides a unified framework for advancing abstract visual reasoning in multimodal research by isolating distinct reasoning abilities and revealing current model limitations.

Abstract: Vision-Language Models (VLMs) have achieved remarkable progress across tasks such as visual question answering and image captioning. Yet, the extent to which these models perform visual reasoning as opposed to relying on linguistic priors remains unclear. To address this, we introduce VisRes Bench, a benchmark designed to study visual reasoning in naturalistic settings without contextual language supervision. Analyzing model behavior across three levels of complexity, we uncover clear limitations in perceptual and relational visual reasoning capacities. VisRes isolates distinct reasoning abilities across its levels. Level 1 probes perceptual completion and global image matching under perturbations such as blur, texture changes, occlusion, and rotation; Level 2 tests rule-based inference over a single attribute (e.g., color, count, orientation); and Level 3 targets compositional reasoning that requires integrating multiple visual attributes. Across more than 19,000 controlled task images, we find that state-of-the-art VLMs perform near random under subtle perceptual perturbations, revealing limited abstraction beyond pattern recognition. We conclude by discussing how VisRes provides a unified framework for advancing abstract visual reasoning in multimodal research.

Method: Uses everyday wearables with multimodal teacher-student framework integrating visual cues from egocentric cameras (forward-facing + two downward-facing) with inertial signals from consumer devices. Trained on Ego-Elec dataset (9-hour real-world data covering 56 daily activities across 17 environments) rather than synthetic data.

Result: Outperforms baseline models, demonstrating effectiveness for practical full-body motion estimation. Eliminates sim-to-real gap that constrained prior work.

Conclusion: EveryWear provides a lightweight, practical solution for human motion capture using everyday wearables without calibration, enabling more accessible adoption in daily life applications.

Abstract: While on-body device-based human motion estimation is crucial for applications such as XR interaction, existing methods often suffer from poor wearability, expensive hardware, and cumbersome calibration, which hinder their adoption in daily life. To address these challenges, we present EveryWear, a lightweight and practical human motion capture approach based entirely on everyday wearables: a smartphone, smartwatch, earbuds, and smart glasses equipped with one forward-facing and two downward-facing cameras, requiring no explicit calibration before use. We introduce Ego-Elec, a 9-hour real-world dataset covering 56 daily activities across 17 diverse indoor and outdoor environments, with ground-truth 3D annotations provided by the motion capture (MoCap), to facilitate robust research and benchmarking in this direction. Our approach employs a multimodal teacher-student framework that integrates visual cues from egocentric cameras with inertial signals from consumer devices. By training directly on real-world data rather than synthetic data, our model effectively eliminates the sim-to-real gap that constrains prior work. Experiments demonstrate that our method outperforms baseline models, validating its effectiveness for practical full-body motion estimation.

Method: A task-agnostic mechanism that trains LMMs to discover and use visual reasoning tokens without explicit supervision. These tokens attend globally and re-encode images in a task-adaptive way to extract relevant visual information.

Result: The approach outperforms direct fine-tuning and achieves state-of-the-art results on diverse vision-centric tasks, including those where intermediate abstractions are hard to specify, while generalizing to multi-task instruction tuning.

Conclusion: The proposed unsupervised visual token discovery method enables LMMs to better handle visual reasoning tasks without hand-crafted supervision, addressing a critical limitation of current text-centric multimodal models.

Abstract: While Large Multimodal Models (LMMs) have made significant progress, they remain largely text-centric, relying on language as their core reasoning modality. As a result, they are limited in their ability to handle reasoning tasks that are predominantly visual. Recent approaches have sought to address this by supervising intermediate visual steps with helper images, depth maps, or image crops. However, these strategies impose restrictive priors on what “useful” visual abstractions look like, add heavy annotation costs, and struggle to generalize across tasks. To address this critical limitation, we propose a task-agnostic mechanism that trains LMMs to discover and use visual reasoning tokens without explicit supervision. These tokens attend globally and re-encode the image in a task-adaptive way, enabling the model to extract relevant visual information without hand-crafted supervision. Our approach outperforms direct fine-tuning and achieves state-of-the-art results on a diverse range of vision-centric tasks – including those where intermediate abstractions are hard to specify – while also generalizing to multi-task instruction tuning.

Method: Lightweight two-stage pipeline: 1) BM25-based candidate filtering using event-centric entity extraction to capture temporal/contextual signals, 2) BEiT-3 models for deep multimodal semantic matching and reranking.

Result: Achieves mean average precision of 0.559 on OpenEvents v1 benchmark, substantially outperforming prior baselines.

Conclusion: Combining event-guided filtering with long-text vision-language modeling is effective for accurate and efficient retrieval in complex real-world scenarios.

Abstract: Retrieving images from natural language descriptions is a core task at the intersection of computer vision and natural language processing, with wide-ranging applications in search engines, media archiving, and digital content management. However, real-world image-text retrieval remains challenging due to vague or context-dependent queries, linguistic variability, and the need for scalable solutions. In this work, we propose a lightweight two-stage retrieval pipeline that leverages event-centric entity extraction to incorporate temporal and contextual signals from real-world captions. The first stage performs efficient candidate filtering using BM25 based on salient entities, while the second stage applies BEiT-3 models to capture deep multimodal semantics and rerank the results. Evaluated on the OpenEvents v1 benchmark, our method achieves a mean average precision of 0.559, substantially outperforming prior baselines. These results highlight the effectiveness of combining event-guided filtering with long-text vision-language modeling for accurate and efficient retrieval in complex, real-world scenarios. Our code is available at https://github.com/PhamPhuHoa-23/Event-Based-Image-Retrieval

Method: Three-module framework: (1) Text Segment Extraction decomposes textual descriptions into temporally ordered phrases representing atomic actions; (2) Motion Segment Extraction partitions motion sequences into corresponding segments; (3) Fine-grained Text-Motion Alignment aligns text and motion segments using contrastive learning in a shared embedding space.

Result: SegMo improves strong baselines on two widely used datasets, achieving TOP 1 score of 0.553 on HumanML3D test set. The learned shared embedding space enables applications to retrieval-style tasks like motion grounding and motion-to-text retrieval.

Conclusion: Segment-level alignment provides finer-grained text-motion correspondence, improving generation quality and enabling cross-modal retrieval applications. The decomposition into atomic action segments is a promising direction for text-conditioned motion generation.

Abstract: Generating 3D human motions from textual descriptions is an important research problem with broad applications in video games, virtual reality, and augmented reality. Recent methods align the textual description with human motion at the sequence level, neglecting the internal semantic structure of modalities. However, both motion descriptions and motion sequences can be naturally decomposed into smaller and semantically coherent segments, which can serve as atomic alignment units to achieve finer-grained correspondence. Motivated by this, we propose SegMo, a novel Segment-aligned text-conditioned human Motion generation framework to achieve fine-grained text-motion alignment. Our framework consists of three modules: (1) Text Segment Extraction, which decomposes complex textual descriptions into temporally ordered phrases, each representing a simple atomic action; (2) Motion Segment Extraction, which partitions complete motion sequences into corresponding motion segments; and (3) Fine-grained Text-Motion Alignment, which aligns text and motion segments with contrastive learning. Extensive experiments demonstrate that SegMo improves the strong baseline on two widely used datasets, achieving an improved TOP 1 score of 0.553 on the HumanML3D test set. Moreover, thanks to the learned shared embedding space for text and motion segments, SegMo can also be applied to retrieval-style tasks such as motion grounding and motion-to-text retrieval.

Method: Two-stage architecture: 1) Causal Chain Extractor (CCE) generates natural language causal chains from video-question pairs, 2) Causal Chain-Driven Answerer (CCDA) derives answers grounded in these chains. Introduces scalable method for generating annotated causal chains from existing datasets.

Result: Outperforms state-of-the-art models on three large-scale benchmarks. Creates human-verified causal chains for 46K samples. Introduces CauCo metric for causality-oriented captioning. Shows substantial gains in explainability, user trust, and generalization.

Conclusion: The modular approach with explicit causal chains enables transparent and logically coherent inference, positioning CCE as a reusable causal reasoning engine across diverse domains while improving both performance and interpretability.

Abstract: Existing Causal-Why Video Question Answering (VideoQA) models often struggle with higher-order reasoning, relying on opaque, monolithic pipelines that entangle video understanding, causal inference, and answer generation. These black-box approaches offer limited interpretability and tend to depend on shallow heuristics. We propose a novel, modular paradigm that explicitly decouples causal reasoning from answer generation, introducing natural language causal chains as interpretable intermediate representations. Inspired by human cognitive models, these structured cause-effect sequences bridge low-level video content with high-level causal reasoning, enabling transparent and logically coherent inference. Our two-stage architecture comprises a Causal Chain Extractor (CCE) that generates causal chains from video-question pairs, and a Causal Chain-Driven Answerer (CCDA) that derives answers grounded in these chains. To address the lack of annotated reasoning traces, we introduce a scalable method for generating accurate causal chains from existing datasets. We construct human verified causal chains for 46K samples. We also propose CauCo, a new evaluation metric for causality-oriented captioning. Experiments on three large-scale benchmarks demonstrate that our approach not only outperforms state-of-the-art models, but also yields substantial gains in explainability, user trust, and generalization – positioning the CCE as a reusable causal reasoning engine across diverse domains. Project page: https://paritoshparmar.github.io/chainreaction/

Method: Three-dimensional approach: (1) Lightweight intermediate-conditioning mechanism in DiT architecture with Adaptive Tuning for arbitrary-frame control; (2) High-quality dataset curation with Visual Expression SFT and Tailored DPO for subject motion rationality and transition smoothness; (3) Segment-wise Auto-Regressive (SAR) inference for memory-efficient long sequence generation.

Result: Extensive experiments show DreaMontage achieves visually striking and seamlessly coherent one-shot effects while maintaining computational efficiency, enabling transformation of fragmented visual materials into cohesive cinematic experiences.

Conclusion: DreaMontage provides a comprehensive framework for generating high-quality one-shot videos from diverse user inputs, overcoming limitations of existing methods through innovative architectural, training, and inference strategies.

Abstract: The “one-shot” technique represents a distinct and sophisticated aesthetic in filmmaking. However, its practical realization is often hindered by prohibitive costs and complex real-world constraints. Although emerging video generation models offer a virtual alternative, existing approaches typically rely on naive clip concatenation, which frequently fails to maintain visual smoothness and temporal coherence. In this paper, we introduce DreaMontage, a comprehensive framework designed for arbitrary frame-guided generation, capable of synthesizing seamless, expressive, and long-duration one-shot videos from diverse user-provided inputs. To achieve this, we address the challenge through three primary dimensions. (i) We integrate a lightweight intermediate-conditioning mechanism into the DiT architecture. By employing an Adaptive Tuning strategy that effectively leverages base training data, we unlock robust arbitrary-frame control capabilities. (ii) To enhance visual fidelity and cinematic expressiveness, we curate a high-quality dataset and implement a Visual Expression SFT stage. In addressing critical issues such as subject motion rationality and transition smoothness, we apply a Tailored DPO scheme, which significantly improves the success rate and usability of the generated content. (iii) To facilitate the production of extended sequences, we design a Segment-wise Auto-Regressive (SAR) inference strategy that operates in a memory-efficient manner. Extensive experiments demonstrate that our approach achieves visually striking and seamlessly coherent one-shot effects while maintaining computational efficiency, empowering users to transform fragmented visual materials into vivid, cohesive one-shot cinematic experiences.

Method: Created the Remote Sensing Change Caption (RSCC) dataset - a large-scale benchmark with 62,351 pre-/post-disaster image pairs covering earthquakes, floods, wildfires, and other disasters, paired with rich, human-like change captions.

Result: RSCC enables robust training and evaluation of vision-language models for disaster-aware bi-temporal understanding. The dataset facilitates detailed disaster-related analysis and supports more accurate, interpretable vision-language applications.

Conclusion: RSCC bridges the temporal and semantic divide in remote sensing data, paving the way for scalable vision-language applications in disaster monitoring. The dataset and code are publicly available.

Abstract: Remote sensing is critical for disaster monitoring, yet existing datasets lack temporal image pairs and detailed textual annotations. While single-snapshot imagery dominates current resources, it fails to capture dynamic disaster impacts over time. To address this gap, we introduce the Remote Sensing Change Caption (RSCC) dataset, a large-scale benchmark comprising 62,351 pre-/post-disaster image pairs (spanning earthquakes, floods, wildfires, and more) paired with rich, human-like change captions. By bridging the temporal and semantic divide in remote sensing data, RSCC enables robust training and evaluation of vision-language models for disaster-aware bi-temporal understanding. Our results highlight RSCC’s ability to facilitate detailed disaster-related analysis, paving the way for more accurate, interpretable, and scalable vision-language applications in remote sensing. Code and dataset are available at https://github.com/Bili-Sakura/RSCC.

Method: Unified framework with dual-pathway DINOv2 encoder and feature distribution alignment to handle incomplete modalities. Includes Intrinsic Normal Prototypes (INPs) extractor and INP-guided decoder that reconstructs only normal anatomical patterns while amplifying abnormal deviations. Uses randomized modality masking and indirect feature completion during training.

Result: Consistently surpasses state-of-the-art industrial and medical AD baselines across 7 modality combinations on BraTS2018, MU-Glioma-Post, and Pretreat-MetsToBrain-Masks datasets, achieving superior generalization.

Conclusion: Establishes a scalable paradigm for multimodal medical anomaly detection under real-world, imperfect modality conditions, enabling robust performance with arbitrary modality availability without retraining.

Abstract: Reliable anomaly detection in brain MRI remains challenging due to the scarcity of annotated abnormal cases and the frequent absence of key imaging modalities in real clinical workflows. Existing single-class or multi-class anomaly detection (AD) models typically rely on fixed modality configurations, require repetitive training, or fail to generalize to unseen modality combinations, limiting their clinical scalability. In this work, we present a unified Any-Modality AD framework that performs robust anomaly detection and localization under arbitrary MRI modality availability. The framework integrates a dual-pathway DINOv2 encoder with a feature distribution alignment mechanism that statistically aligns incomplete-modality features with full-modality representations, enabling stable inference even with severe modality dropout. To further enhance semantic consistency, we introduce an Intrinsic Normal Prototypes (INPs) extractor and an INP-guided decoder that reconstruct only normal anatomical patterns while naturally amplifying abnormal deviations. Through randomized modality masking and indirect feature completion during training, the model learns to adapt to all modality configurations without re-training. Extensive experiments on BraTS2018, MU-Glioma-Post, and Pretreat-MetsToBrain-Masks demonstrate that our approach consistently surpasses state-of-the-art industrial and medical AD baselines across 7 modality combinations, achieving superior generalization. This study establishes a scalable paradigm for multimodal medical AD under real-world, imperfect modality conditions. Our source code is available at https://github.com/wuchangw/AnyAD.

Method: Develop scalable 2DGS pipeline with structured initialization, luminance-aware pruning, and batched CUDA kernels. Adapt CLIP to 2DGS using frozen RGB transformer backbone with lightweight splat-aware input stem and perceiver resampler, training only 9.7-13.8% of parameters.

Result: Achieved over 90x faster fitting and ~97% GPU utilization compared to prior implementations. GS encoders yield competitive zero-shot performance on 38 CLIP benchmark datasets while compressing inputs 3x to 23.5x relative to pixels.

Conclusion: 2DGS established as viable multimodal substrate that addresses architectural bottlenecks, opening path toward representations that are both semantically powerful and transmission-efficient for edge-cloud learning.

Abstract: Modern vision language pipelines are driven by RGB vision encoders trained on massive image text corpora. While these pipelines have enabled impressive zero-shot capabilities and strong transfer across tasks, they still inherit two structural inefficiencies from the pixel domain: (i) transmitting dense RGB images from edge devices to the cloud is energy-intensive and costly, and (ii) patch-based tokenization explodes sequence length, stressing attention budgets and context limits. We explore 2D Gaussian Splatting (2DGS) as an alternative visual substrate for alignment: a compact, spatially adaptive representation that parameterizes images by a set of colored anisotropic Gaussians. We develop a scalable 2DGS pipeline with structured initialization, luminance-aware pruning, and batched CUDA kernels, achieving over 90x faster fitting and about 97% GPU utilization compared to prior implementations. We further adapt contrastive language-image pre-training (CLIP) to 2DGS by reusing a frozen RGB-based transformer backbone with a lightweight splat-aware input stem and a perceiver resampler, training only 9.7% to 13.8% of the total parameters. On a 12.8M dataset from DataComp, GS encoders yield competitive zero-shot performance on 38 datasets from the CLIP benchmark while compressing inputs 3x to 23.5x relative to pixels. Our results establish 2DGS as a viable multimodal substrate, pinpoint architectural bottlenecks, and open a path toward representations that are both semantically powerful and transmission-efficient for edge-cloud learning.

Method: Attention-Conditional Diffusion (ACD) uses attention supervision to align the model’s attention maps with external control signals. It introduces sparse 3D-aware object layouts as conditioning signals, a Layout ControlNet, and an automated annotation pipeline for scalable layout integration.

Result: Extensive experiments on benchmark video generation datasets show ACD delivers superior alignment with conditioning inputs while preserving temporal coherence and visual fidelity.

Conclusion: ACD establishes an effective paradigm for conditional video synthesis by enabling direct control through attention supervision, overcoming limitations of existing guidance methods.

Abstract: Controllability is a fundamental requirement in video synthesis, where accurate alignment with conditioning signals is essential. Existing classifier-free guidance methods typically achieve conditioning indirectly by modeling the joint distribution of data and conditions, which often results in limited controllability over the specified conditions. Classifier-based guidance enforces conditions through an external classifier, but the model may exploit this mechanism to raise the classifier score without genuinely satisfying the intended condition, resulting in adversarial artifacts and limited effective controllability. In this paper, we propose Attention-Conditional Diffusion (ACD), a novel framework for direct conditional control in video diffusion models via attention supervision. By aligning the model’s attention maps with external control signals, ACD achieves better controllability. To support this, we introduce a sparse 3D-aware object layout as an efficient conditioning signal, along with a dedicated Layout ControlNet and an automated annotation pipeline for scalable layout integration. Extensive experiments on benchmark video generation datasets demonstrate that ACD delivers superior alignment with conditioning inputs while preserving temporal coherence and visual fidelity, establishing an effective paradigm for conditional video synthesis.

Method: The method factorizes generation into two stages: 1) Generate coarse sequence at low resolution using a generative model trained on grid images of subsampled frames, leveraging Diffusion Transformer’s self-attention to capture frame correlations, effectively extending a 2D image generator to operate as low-resolution 3D sequence generator without architectural changes. 2) Super-resolve each frame individually to add sequence-independent high-resolution details.

Result: The method achieves superior synthesis quality and improved coherence across sequences compared to existing models. It enables high-fidelity generation of arbitrary-length sequences with increased efficiency in inference time (at least twice-as-fast) and training data usage. It generalizes effectively across diverse data domains without requiring additional priors or supervision.

Conclusion: The proposed factorization approach overcomes key limitations of current SoTA image sequence generation methods, offering a more effective representation that consistently outperforms existing methods in both quality and inference speed across diverse datasets.

Abstract: Modern deep learning methods typically treat image sequences as large tensors of sequentially stacked frames. However, is this straightforward representation ideal given the current state-of-the-art (SoTA)? In this work, we address this question in the context of generative models and aim to devise a more effective way of modeling image sequence data. Observing the inefficiencies and bottlenecks of current SoTA image sequence generation methods, we showcase that rather than working with large tensors, we can improve the generation process by factorizing it into first generating the coarse sequence at low resolution and then refining the individual frames at high resolution. We train a generative model solely on grid images comprising subsampled frames. Yet, we learn to generate image sequences, using the strong self-attention mechanism of the Diffusion Transformer (DiT) to capture correlations between frames. In effect, our formulation extends a 2D image generator to operate as a low-resolution 3D image-sequence generator without introducing any architectural modifications. Subsequently, we super-resolve each frame individually to add the sequence-independent high-resolution details. This approach offers several advantages and can overcome key limitations of the SoTA in this domain. Compared to existing image sequence generation models, our method achieves superior synthesis quality and improved coherence across sequences. It also delivers high-fidelity generation of arbitrary-length sequences and increased efficiency in inference time and training data usage. Furthermore, our straightforward formulation enables our method to generalize effectively across diverse data domains, which typically require additional priors and supervision to model in a generative context. Our method consistently outperforms SoTA in quality and inference speed (at least twice-as-fast) across datasets.

Method: Proposes SpikeSurgSeg with: 1) surgical-scene masked autoencoding pretraining using layer-wise tube masking for robust spatiotemporal representation learning, and 2) lightweight spike-driven segmentation head for temporally consistent predictions while preserving SNN low-latency characteristics.

Result: Achieves mIoU comparable to SOTA ANN-based models while reducing inference latency by at least 8× and delivering over 20× acceleration relative to foundation-model baselines on EndoVis18 and SurgBleed datasets.

Conclusion: SpikeSurgSeg demonstrates the potential of SNNs for time-critical surgical scene segmentation, enabling real-time deployment on non-GPU platforms while maintaining accuracy comparable to computationally intensive ANN models.

Abstract: Modern surgical systems increasingly rely on intelligent scene understanding to provide timely situational awareness for enhanced intra-operative safety. Within this pipeline, surgical scene segmentation plays a central role in accurately perceiving operative events. Although recent deep learning models, particularly large-scale foundation models, achieve remarkable segmentation accuracy, their substantial computational demands and power consumption hinder real-time deployment in resource-constrained surgical environments. To address this limitation, we explore the emerging SNN as a promising paradigm for highly efficient surgical intelligence. However, their performance is still constrained by the scarcity of labeled surgical data and the inherently sparse nature of surgical video representations. To this end, we propose \textit{SpikeSurgSeg}, the first spike-driven video Transformer framework tailored for surgical scene segmentation with real-time potential on non-GPU platforms. To address the limited availability of surgical annotations, we introduce a surgical-scene masked autoencoding pretraining strategy for SNNs that enables robust spatiotemporal representation learning via layer-wise tube masking. Building on this pretrained backbone, we further adopt a lightweight spike-driven segmentation head that produces temporally consistent predictions while preserving the low-latency characteristics of SNNs. Extensive experiments on EndoVis18 and our in-house SurgBleed dataset demonstrate that SpikeSurgSeg achieves mIoU comparable to SOTA ANN-based models while reducing inference latency by at least $8\times$. Notably, it delivers over $20\times$ acceleration relative to most foundation-model baselines, underscoring its potential for time-critical surgical scene segmentation.

Method: Two key contributions: (1) a new large-scale dataset of image-sketch-instruction triplets for both pretraining and instruction tuning, and (2) O3SLM, an LVLM trained on this dataset. The model is evaluated on multiple sketch-based tasks including object localization, counting, image retrieval (SBIR and fine-grained SBIR), and visual question answering (VQA).

Result: O3SLM achieves state-of-the-art performance on comprehensive evaluations across multiple sketch-based tasks, substantially outperforming existing LVLMs in sketch comprehension and reasoning. The evaluation incorporates three existing sketch datasets (QuickDraw!, Sketchy, Tu Berlin) along with their generated SketchVCL dataset.

Conclusion: The proposed approach successfully addresses the bottleneck in LVLM sketch comprehension by providing both a large-scale training dataset and a specialized model, demonstrating significant improvements in sketch understanding capabilities compared to existing methods.

Abstract: While Large Vision Language Models (LVLMs) are increasingly deployed in real-world applications, their ability to interpret abstract visual inputs remains limited. Specifically, they struggle to comprehend hand-drawn sketches, a modality that offers an intuitive means of expressing concepts that are difficult to describe textually. We identify the primary bottleneck as the absence of a large-scale dataset that jointly models sketches, photorealistic images, and corresponding natural language instructions. To address this, we present two key contributions: (1) a new, large-scale dataset of image-sketch-instruction triplets designed to facilitate both pretraining and instruction tuning, and (2) O3SLM, an LVLM trained on this dataset. Comprehensive evaluations on multiple sketch-based tasks: (a) object localization, (b) counting, (c) image retrieval i.e., (SBIR and fine-grained SBIR), and (d) visual question answering (VQA); while incorporating the three existing sketch datasets, namely QuickDraw!, Sketchy, and Tu Berlin, along with our generated SketchVCL dataset, show that O3SLM achieves state-of-the-art performance, substantially outperforming existing LVLMs in sketch comprehension and reasoning.

Method: Models row and column transitions as 1D signals, processes them using Gaussian convolution with progressively increasing variances, applies statistical thresholding to suppress noise while preserving stable structural edges, and maps detected signal peaks back to image coordinates to obtain accurate segment boundaries.

Result: Improves Cell-Aware Segmentation Accuracy (CASA) from 67% to 76% on PubLayNet-1M benchmark when using TableNet with PyTesseract OCR. The method is robust to resolution variations through zero-padding and scaling strategies.

Conclusion: The proposed multi-scale signal-processing approach effectively detects table edges from masks, offering improved accuracy and robustness for structured data extraction from tables in challenging document images, producing optimized outputs suitable for downstream analysis.

Abstract: Structured data extraction from tables plays a crucial role in document image analysis for scanned documents and digital archives. Although many methods have been proposed to detect table structures and extract cell contents, accurately identifying table segment boundaries (rows and columns) remains challenging, particularly in low-resolution or noisy images. In many real-world scenarios, table data are incomplete or degraded, limiting the adaptability of transformer-based methods to noisy inputs. Mask-based edge detection techniques have shown greater robustness under such conditions, as their sensitivity can be adjusted through threshold tuning; however, existing approaches typically apply masks directly to images, leading to noise sensitivity, resolution loss, or high computational cost. This paper proposes a novel multi-scale signal-processing method for detecting table edges from table masks. Row and column transitions are modeled as one-dimensional signals and processed using Gaussian convolution with progressively increasing variances, followed by statistical thresholding to suppress noise while preserving stable structural edges. Detected signal peaks are mapped back to image coordinates to obtain accurate segment boundaries. Experimental results show that applying the proposed approach to column edge detection improves Cell-Aware Segmentation Accuracy (CASA) a layout-aware metric evaluating both textual correctness and correct cell placement from 67% to 76% on the PubLayNet-1M benchmark when using TableNet with PyTesseract OCR. The method is robust to resolution variations through zero-padding and scaling strategies and produces optimized structured tabular outputs suitable for downstream analysis.

Method: Created AndroidLens with 571 long-latency tasks across 38 domains, featuring: 1) real-world multi-constraint/multi-goal tasks, 2) static evaluation preserving anomalies and multiple valid paths, 3) dynamic evaluation with milestone-based ATP measurement.

Result: Even best models achieve only 12.7% task success rate and 50.47% Average Task Progress, highlighting significant performance gaps in real-world mobile GUI automation.

Conclusion: AndroidLens reveals major challenges in mobile GUI automation including environmental anomalies, adaptive exploration, and long-term memory retention, providing a more realistic benchmark for future agent development.

Abstract: Graphical user interface (GUI) agents can substantially improve productivity by automating frequently executed long-latency tasks on mobile devices. However, existing evaluation benchmarks are still constrained to limited applications, simple tasks, and coarse-grained metrics. To address this, we introduce AndroidLens, a challenging evaluation framework for mobile GUI agents, comprising 571 long-latency tasks in both Chinese and English environments, each requiring an average of more than 26 steps to complete. The framework features: (1) tasks derived from real-world user scenarios across 38 domains, covering complex types such as multi-constraint, multi-goal, and domain-specific tasks; (2) static evaluation that preserves real-world anomalies and allows multiple valid paths to reduce bias; and (3) dynamic evaluation that employs a milestone-based scheme for fine-grained progress measurement via Average Task Progress (ATP). Our evaluation indicates that even the best models reach only a 12.7% task success rate and 50.47% ATP. We also underscore key challenges in real-world environments, including environmental anomalies, adaptive exploration, and long-term memory retention.

Method: TICON uses a single shared transformer encoder pretrained with a masked modeling objective to simultaneously unify and contextualize representations from diverse tile-level pathology foundation models. It also includes an aggregator pretrained on TICON to form a slide-level foundation model.

Result: TICON-contextualized embeddings significantly improve performance across many tasks, establishing new SOTA results on tile-level benchmarks (HEST-Bench, THUNDER, CATCH) and slide-level benchmarks (Patho-Bench). The slide-level foundation model trained with only 11K WSIs outperforms SOTA models trained with up to 350K WSIs.

Conclusion: TICON successfully addresses the need for contextualized tile embeddings in computational pathology, providing a unified approach that works with any tile-level foundation model and significantly improves performance across diverse tasks while enabling efficient slide-level foundation modeling.

Abstract: The interpretation of small tiles in large whole slide images (WSI) often needs a larger image context. We introduce TICON, a transformer-based tile representation contextualizer that produces rich, contextualized embeddings for ‘‘any’’ application in computational pathology. Standard tile encoder-based pipelines, which extract embeddings of tiles stripped from their context, fail to model the rich slide-level information essential for both local and global tasks. Furthermore, different tile-encoders excel at different downstream tasks. Therefore, a unified model is needed to contextualize embeddings derived from ‘‘any’’ tile-level foundation model. TICON addresses this need with a single, shared encoder, pretrained using a masked modeling objective to simultaneously unify and contextualize representations from diverse tile-level pathology foundation models. Our experiments demonstrate that TICON-contextualized embeddings significantly improve performance across many different tasks, establishing new state-of-the-art results on tile-level benchmarks (i.e., HEST-Bench, THUNDER, CATCH) and slide-level benchmarks (i.e., Patho-Bench). Finally, we pretrain an aggregator on TICON to form a slide-level foundation model, using only 11K WSIs, outperforming SoTA slide-level foundation models pretrained with up to 350K WSIs.

Method: A text-guided token pruning framework that operates after visual encoding and before memory-based propagation. Uses a lightweight routing mechanism integrating local visual context, semantic relevance from object-centric textual descriptions (user-provided or auto-generated), and uncertainty cues to preserve ambiguous/boundary regions. Ranks and retains only the most informative tokens for downstream processing.

Result: Achieves up to 42.50% faster inference and 37.41% lower GPU memory usage compared to unpruned baseline SAM2, while preserving competitive J and F performance across multiple challenging video segmentation benchmarks.

Conclusion: Post-encoder token pruning provides a practical and effective pathway to efficient, prompt-aware video segmentation. Early token selection can improve scalability of transformer-based video segmentation systems for real-time and resource-constrained applications.

Abstract: Segment Anything Model 2 (SAM2), a vision foundation model has significantly advanced in prompt-driven video object segmentation, yet their practical deployment remains limited by the high computational and memory cost of processing dense visual tokens across time. The SAM2 pipelines typically propagate all visual tokens produced by the image encoder through downstream temporal reasoning modules, regardless of their relevance to the target object, resulting in reduced scalability due to quadratic memory attention overhead. In this work, we introduce a text-guided token pruning framework that improves inference efficiency by selectively reducing token density prior to temporal propagation, without modifying the underlying segmentation architecture. Operating after visual encoding and before memory based propagation, our method ranks tokens using a lightweight routing mechanism that integrates local visual context, semantic relevance derived from object-centric textual descriptions (either user-provided or automatically generated), and uncertainty cues that help preserve ambiguous or boundary critical regions. By retaining only the most informative tokens for downstream processing, the proposed approach reduces redundant computation while maintaining segmentation fidelity. Extensive experiments across multiple challenging video segmentation benchmarks demonstrate that post-encoder token pruning provides a practical and effective pathway to efficient, prompt-aware video segmentation, achieving up to 42.50 percent faster inference and 37.41 percent lower GPU memory usage compared to the unpruned baseline SAM2, while preserving competitive J and F performance. These results highlight the potential of early token selection to improve the scalability of transformer-based video segmentation systems for real-time and resource-constrained applications.

Method: Created Streamo-Instruct-465K, a large-scale instruction-following dataset for streaming video understanding with diverse temporal contexts and multi-task supervision. Trained end-to-end through a streamlined pipeline on this dataset.

Result: Streamo exhibits strong temporal reasoning, responsive interaction, and broad generalization across various streaming benchmarks. It bridges the gap between offline video perception models and real-time multimodal assistants.

Conclusion: Streamo represents a step toward unified, intelligent video understanding in continuous video streams, demonstrating versatility across multiple streaming video tasks with a single model.

Abstract: We present Streamo, a real-time streaming video LLM that serves as a general-purpose interactive assistant. Unlike existing online video models that focus narrowly on question answering or captioning, Streamo performs a broad spectrum of streaming video tasks, including real-time narration, action understanding, event captioning, temporal event grounding, and time-sensitive question answering. To develop such versatility, we construct Streamo-Instruct-465K, a large-scale instruction-following dataset tailored for streaming video understanding. The dataset covers diverse temporal contexts and multi-task supervision, enabling unified training across heterogeneous streaming tasks. After training end-to-end on the instruction-following dataset through a streamlined pipeline, Streamo exhibits strong temporal reasoning, responsive interaction, and broad generalization across a variety of streaming benchmarks. Extensive experiments show that Streamo bridges the gap between offline video perception models and real-time multimodal assistants, making a step toward unified, intelligent video understanding in continuous video streams.

Method: Created YearGuessr dataset (55,546 building images with construction years, GPS, page-view counts), framed construction year prediction as ordinal regression, introduced popularity-aware interval accuracy metrics, benchmarked 30+ models including YearCLIP.

Result: VLMs achieve up to 34% higher accuracy on famous buildings, confirming they excel on popular/memorized items but struggle significantly with unrecognized subjects, exposing critical reasoning flaws.

Conclusion: Current VLMs have significant popularity bias and rely on memorization rather than generalizable understanding, highlighting a critical flaw in their reasoning capabilities that needs addressing.

Abstract: We expose a significant popularity bias in state-of-the-art vision-language models (VLMs), which achieve up to 34% higher accuracy on famous buildings compared to ordinary ones, indicating a reliance on memorization over generalizable understanding. To systematically investigate this, we introduce the largest open benchmark for this task: the YearGuessr dataset, a collection of 55,546 building images with multi-modal attributes from 157 countries, annotated with continuous ordinal labels of their construction year (1001-2024), GPS data, and page-view counts as a proxy for popularity. Using this dataset, we frame the construction year prediction task as ordinal regression and introduce popularity-aware interval accuracy metrics to quantify this bias. Our resulting benchmark of 30+ models, including our YearCLIP model, confirms that VLMs excel on popular, memorized items but struggle significantly with unrecognized subjects, exposing a critical flaw in their reasoning capabilities. Project page: https://sytwu.github.io/BeyondMemo/

Method: HiStream uses three compression strategies: 1) Spatial Compression - denoising at low resolution first then refining at high resolution with cached features; 2) Temporal Compression - chunk-by-chunk processing with fixed-size anchor cache for stable inference; 3) Timestep Compression - applying fewer denoising steps to subsequent cache-conditioned chunks.

Result: On 1080p benchmarks, HiStream achieves state-of-the-art visual quality with up to 76.2x faster denoising than Wan2.1 baseline and negligible quality loss. HiStream+ (with all three optimizations) achieves 107.5x acceleration, offering compelling speed-quality trade-off.

Conclusion: HiStream makes high-resolution video generation practical and scalable by dramatically reducing computational requirements while maintaining visual quality, addressing the fundamental bottleneck in diffusion-based video generation.

Abstract: High-resolution video generation, while crucial for digital media and film, is computationally bottlenecked by the quadratic complexity of diffusion models, making practical inference infeasible. To address this, we introduce HiStream, an efficient autoregressive framework that systematically reduces redundancy across three axes: i) Spatial Compression: denoising at low resolution before refining at high resolution with cached features; ii) Temporal Compression: a chunk-by-chunk strategy with a fixed-size anchor cache, ensuring stable inference speed; and iii) Timestep Compression: applying fewer denoising steps to subsequent, cache-conditioned chunks. On 1080p benchmarks, our primary HiStream model (i+ii) achieves state-of-the-art visual quality while demonstrating up to 76.2x faster denoising compared to the Wan2.1 baseline and negligible quality loss. Our faster variant, HiStream+, applies all three optimizations (i+ii+iii), achieving a 107.5x acceleration over the baseline, offering a compelling trade-off between speed and quality, thereby making high-resolution video generation both practical and scalable.

Method: DenseVLM leverages pre-trained VLMs to retrieve categories for unlabeled regions and decouples interference between foreground and background features to learn unbiased region-language alignment.

Result: DenseVLM can directly replace original VLMs in open-vocabulary object detection and image segmentation methods, leading to notable performance improvements. It also shows promising zero-shot scalability when trained on more extensive and diverse datasets.

Conclusion: DenseVLM effectively addresses foreground bias in VLMs for dense prediction tasks, enabling better adaptation to local regions without extensive annotations while maintaining scalability.

Abstract: Pre-trained vision-language models (VLMs), such as CLIP, have demonstrated impressive zero-shot recognition capability, but still underperform in dense prediction tasks. Self-distillation recently is emerging as a promising approach for fine-tuning VLMs to better adapt to local regions without requiring extensive annotations. However, previous state-of-the-art approaches often suffer from significant `foreground bias’, where models tend to wrongly identify background regions as foreground objects. To alleviate this issue, we propose DenseVLM, a framework designed to learn unbiased region-language alignment from powerful pre-trained VLM representations. To alleviate this issue, we propose DenseVLM, a framework designed to learn unbiased region-language alignment from powerful pre-trained VLM representations. DenseVLM leverages the pre-trained VLM to retrieve categories for unlabeled regions and then decouples the interference between foreground and background features. We show that DenseVLM can directly replace the original VLM in open-vocabulary object detection and image segmentation methods, leading to notable performance improvements. Furthermore, it exhibits promising zero-shot scalability when training on more extensive and diverse datasets. Our code is available at https://github.com/HVision-NKU/DenseVLM.

Method: Uses point maps (rasterized 3D scene coordinates) as conditioning signal for pre-trained 2D diffusion models, with reference attention layers and ControlNet for point map features to guide image generation while respecting geometric fidelity.

Result: Achieves high-quality generation with flexibility over point map conditioning signals (dense depth maps or sparse LiDAR points) and can distill to 3D representations like 3D Gaussian Splatting for improved view extrapolation.

Conclusion: PointmapDiff effectively addresses novel view synthesis in urban driving scenes by leveraging geometric priors through point map conditioning of diffusion models, enabling accurate and consistent results across varying viewpoints.

Abstract: Synthesizing extrapolated views remains a difficult task, especially in urban driving scenes, where the only reliable sources of data are limited RGB captures and sparse LiDAR points. To address this problem, we present PointmapDiff, a framework for novel view synthesis that utilizes pre-trained 2D diffusion models. Our method leverages point maps (i.e., rasterized 3D scene coordinates) as a conditioning signal, capturing geometric and photometric priors from the reference images to guide the image generation process. With the proposed reference attention layers and ControlNet for point map features, PointmapDiff can generate accurate and consistent results across varying viewpoints while respecting geometric fidelity. Experiments on real-life driving data demonstrate that our method achieves high-quality generation with flexibility over point map conditioning signals (e.g., dense depth map or even sparse LiDAR points) and can be used to distill to 3D representations such as 3D Gaussian Splatting for improving view extrapolation.

Method: Proposes BevSplat that represents each ground image pixel as a 3D Gaussian primitive with semantic and spatial features. These primitives are synthesized into a Bird’s-Eye View (BEV) feature map for relative pose estimation. For panoramic query images, introduces an icosphere-based supervision strategy for the Gaussian primitives.

Result: Experimental validation on KITTI and VIGOR datasets (with both pinhole and panoramic query images) shows BevSplat significantly improves localization accuracy over prior approaches.

Conclusion: BevSplat effectively resolves height ambiguity in cross-view localization through feature-based Gaussian primitives, achieving superior performance on standard benchmarks and handling both pinhole and panoramic query images effectively.

Abstract: This paper addresses the problem of weakly supervised cross-view localization, where the goal is to estimate the pose of a ground camera relative to a satellite image with noisy ground truth annotations. A common approach to bridge the cross-view domain gap for pose estimation is Bird’s-Eye View (BEV) synthesis. However, existing methods struggle with height ambiguity due to the lack of depth information in ground images and satellite height maps. Previous solutions either assume a flat ground plane or rely on complex models, such as cross-view transformers. We propose BevSplat, a novel method that resolves height ambiguity by using feature-based Gaussian primitives. Each pixel in the ground image is represented by a 3D Gaussian with semantic and spatial features, which are synthesized into a BEV feature map for relative pose estimation. Additionally, to address challenges with panoramic query images, we introduce an icosphere-based supervision strategy for the Gaussian primitives. We validate our method on the widely used KITTI and VIGOR datasets, which include both pinhole and panoramic query images. Experimental results show that BevSplat significantly improves localization accuracy over prior approaches.

Method: Proposes a VLM-based pseudo-labeling approach with state-change dynamics constraints, and introduces the WhereToChange benchmark built on in-the-wild Internet videos for evaluation.

Result: Experiments on two datasets show current SOTA VLMs and video segmentation methods struggle with this task, validating its difficulty. The proposed model shows promise in localizing where and how fast objects change in videos.

Conclusion: Spatial OSC segmentation represents a new frontier for video understanding that challenges current methods and has practical applications for tracking activity progress in robotics.

Abstract: Object state changes in video reveal critical cues about human and agent activity. However, existing methods are limited to temporal localization of when the object is in its initial state (e.g., cheese block) versus when it has completed a state change (e.g., grated cheese), offering no insight into where the change is unfolding. We propose to deepen the problem by introducing the spatially-progressing object state change segmentation task. The goal is to segment at the pixel-level those regions of an object that are actionable and those that are transformed. We show that state-of-the-art VLMs and video segmentation methods struggle at this task, underscoring its difficulty and novelty. As an initial baseline, we design a VLM-based pseudo-labeling approach, state-change dynamics constraints, and a novel WhereToChange benchmark built on in-the-wild Internet videos. Experiments on two datasets validate both the challenge of the new task as well as the promise of our model for localizing exactly where and how fast objects are changing in video. We further demonstrate useful implications for tracking activity progress to benefit robotic agents. Overall, our work positions spatial OSC segmentation as a new frontier task for video understanding: one that challenges current SOTA methods and invites the community to build more robust, state-change-sensitive representations. Project page: https://vision.cs.utexas.edu/projects/spoc-spatially-progressing-osc

Method: Proposes SGSASR network with two key components: 1) Spacecraft Core Region Recognition Block (SCRRB) using pre-trained saliency detection to identify core regions, and 2) Adaptive-Weighted Feature Fusion Enhancement Mechanism (AFFEM) that selectively aggregates core region features with general features using dynamic weights.

Result: Experimental results show SGSASR outperforms state-of-the-art approaches in spacecraft image super-resolution.

Conclusion: The proposed saliency-guided approach effectively enhances spacecraft image super-resolution by focusing on important regions while suppressing background noise, demonstrating superior performance over existing methods.

Abstract: Spacecraft image super-resolution seeks to enhance low-resolution spacecraft images into high-resolution ones. Although existing arbitrary-scale super-resolution methods perform well on general images, they tend to overlook the difference in features between the spacecraft core region and the large black space background, introducing irrelevant noise. In this paper, we propose a salient region-guided spacecraft image arbitrary-scale super-resolution network (SGSASR), which uses features from the spacecraft core salient regions to guide latent modulation and achieve arbitrary-scale super-resolution. Specifically, we design a spacecraft core region recognition block (SCRRB) that identifies the core salient regions in spacecraft images using a pre-trained saliency detection model. Furthermore, we present an adaptive-weighted feature fusion enhancement mechanism (AFFEM) to selectively aggregate the spacecraft core region features with general image features by dynamic weight parameter to enhance the response of the core salient regions. Experimental results demonstrate that the proposed SGSASR outperforms state-of-the-art approaches.

Method: Three-stage framework: (1) Perception converts visual information into multi-level textual representations, (2) Search iteratively integrates cross-domain knowledge to resolve ambiguity, and (3) Reasoning generates context-aligned image implications via explicit reasoning. Uses lightweight GPT-4o-mini model.

Result: Achieves SOTA performance compared to 15+ MLLMs on English image implication benchmark, huge improvement on Chinese benchmark, comparable with Gemini-3.0-pro on MCQ, outperforms GPT-4o by 36.7% on OSQ. Shows effective generalization to general VQA and visual reasoning tasks.

Conclusion: LAD framework advances image implication understanding by addressing contextual gaps through human-inspired cognitive processes, providing new insights for vision-language reasoning and human-AI interaction while demonstrating strong performance and generalization capabilities.

Abstract: Metaphorical comprehension in images remains a critical challenge for AI systems, as existing models struggle to grasp the nuanced cultural, emotional, and contextual implications embedded in visual content. While multimodal large language models (MLLMs) excel in general Visual Question Answer (VQA) tasks, they struggle with a fundamental limitation on image implication tasks: contextual gaps that obscure the relationships between different visual elements and their abstract meanings. Inspired by the human cognitive process, we propose Let Androids Dream (LAD), a novel framework for image implication understanding and reasoning. LAD addresses contextual missing through the three-stage framework: (1) Perception: converting visual information into rich and multi-level textual representations, (2) Search: iteratively searching and integrating cross-domain knowledge to resolve ambiguity, and (3) Reasoning: generating context-alignment image implication via explicit reasoning. Our framework with the lightweight GPT-4o-mini model achieves SOTA performance compared to 15+ MLLMs on English image implication benchmark and a huge improvement on Chinese benchmark, performing comparable with the Gemini-3.0-pro model on Multiple-Choice Question (MCQ) and outperforms the GPT-4o model 36.7% on Open-Style Question (OSQ). Generalization experiments also show that our framework can effectively benefit general VQA and visual reasoning tasks. Additionally, our work provides new insights into how AI can more effectively interpret image implications, advancing the field of vision-language reasoning and human-AI interaction. Our project is publicly available at https://github.com/MING-ZCH/Let-Androids-Dream-of-Electric-Sheep.

Method: Two main contributions: 1) Stable reference model update strategy that relaxes the frozen reference model constraint, encouraging exploration while maintaining stability through regularization; 2) Timestep-aware training strategy to mitigate reward scale imbalance across different denoising timesteps.

Result: Experimental results show the approach improves performance of state-of-the-art methods on human preference evaluation benchmarks, demonstrating effectiveness across various preference optimization algorithms.

Conclusion: The proposed orthogonal enhancements to diffusion preference optimization successfully address exploration limitations and timestep reward imbalance, leading to better alignment of text-to-image models with human preferences.

Abstract: Aligning text-to-image (T2I) diffusion models with human preferences has emerged as a critical research challenge. While recent advances in this area have extended preference optimization techniques from large language models (LLMs) to the diffusion setting, they often struggle with limited exploration. In this work, we propose a novel and orthogonal approach to enhancing diffusion-based preference optimization. First, we introduce a stable reference model update strategy that relaxes the frozen reference model, encouraging exploration while maintaining a stable optimization anchor through reference model regularization. Second, we present a timestep-aware training strategy that mitigates the reward scale imbalance problem across timesteps. Our method can be integrated into various preference optimization algorithms. Experimental results show that our approach improves the performance of state-of-the-art methods on human preference evaluation benchmarks. The code is available at the Github: https://github.com/kaist-cvml/RethinkingDPO_Diffusion_Models.

Method: Two-stage framework: 1) Robust pre-processing pipeline to remove artifacts and reduce visual differences, 2) Morphology Learning Stage using scan-like synthetic data followed by Task-Specific Adaptation Stage fine-tuned on photo-like target data.

Result: Achieved 1st place in British Heart Foundation Challenge with macro-AUROC of 0.9677 for classifying five common ECG findings, outperforming single-source training baselines.

Conclusion: Incorporating morphology learning from heterogeneous synthetic data sources provides a more robust and generalizable paradigm than conventional single-source training for ECG image interpretation.

Abstract: In real-world clinical practice, electrocardiograms (ECGs) are often captured and shared as photographs. However, publicly available ECG data, and thus most related research, relies on digital signals. This has led to a disconnect in which computer assisted interpretation of ECG cannot easily be applied to ECG images. The emergence of high-fidelity synthetic data generators has introduced practical alternatives by producing realistic, photo-like, ECG images derived from the digital signal that could help narrow this divide. To address this, we propose a novel knowledge augmentation framework that uses synthetic data generated from multiple sources to provide generalisable and accurate interpretation of ECG photographs. Our framework features two key contributions. First, we introduce a robust pre-processing pipeline designed to remove background artifacts and reduces visual differences between images. Second, we implement a two-stage training strategy: a Morphology Learning Stage, where the model captures broad morphological features from visually different, scan-like synthetic data, followed by a Task-Specific Adaptation Stage, where the model is fine-tuned on the photo-like target data. We tested the model on the British Heart Foundation Challenge dataset, to classify five common ECG findings: myocardial infarction (MI), atrial fibrillation, hypertrophy, conduction disturbance, and ST/T changes. Our approach, built upon the ConvNeXt backbone, outperforms a single-source training baseline and achieved \textbf{1st} place in the challenge with an macro-AUROC of \textbf{0.9677}. These results suggest that incorporating morphology learning from heterogeneous sources offers a more robust and generalizable paradigm than conventional single-source training.

Method: 1) Use visual geometry grounded transformer to recover intrinsic/extrinsic camera parameters and dense 3D reconstruction; 2) Reproject dense point cloud onto reference view for pixel-wise alignment; 3) Apply point-conditioned image diffusion module to refine artifacts like holes and noise.

Result: PIS3R outperforms existing methods qualitatively and quantitatively, providing accurate stitching for images with very large parallax while preserving geometric integrity for downstream 3D vision tasks like SfM.

Conclusion: The proposed PIS3R solution effectively addresses the challenge of stitching images with very large parallax through deep 3D reconstruction, offering robust performance and maintaining geometric fidelity for practical 3D vision applications.

Abstract: Image stitching aim to align two images taken from different viewpoints into one seamless, wider image. However, when the 3D scene contains depth variations and the camera baseline is significant, noticeable parallax occurs-meaning the relative positions of scene elements differ substantially between views. Most existing stitching methods struggle to handle such images with large parallax effectively. To address this challenge, in this paper, we propose an image stitching solution called PIS3R that is robust to very large parallax based on the novel concept of deep 3D reconstruction. First, we apply visual geometry grounded transformer to two input images with very large parallax to obtain both intrinsic and extrinsic parameters, as well as the dense 3D scene reconstruction. Subsequently, we reproject reconstructed dense point cloud onto a designated reference view using the recovered camera parameters, achieving pixel-wise alignment and generating an initial stitched image. Finally, to further address potential artifacts such as holes or noise in the initial stitching, we propose a point-conditioned image diffusion module to obtain the refined result.Compared with existing methods, our solution is very large parallax tolerant and also provides results that fully preserve the geometric integrity of all pixels in the 3D photogrammetric context, enabling direct applicability to downstream 3D vision tasks such as SfM. Experimental results demonstrate that the proposed algorithm provides accurate stitching results for images with very large parallax, and outperforms the existing methods qualitatively and quantitatively.

Method: Uses thermal and RGB images together, leveraging reduced scattering in thermal images to see through smoke. Built on 3D Gaussian splatting to fuse information from both modalities and decompose scenes into smoke and non-smoke components.

Result: Validated on synthetic data and a new real-world smoke dataset with RGB and thermal images. Handles broad range of smoke densities and adapts to temporally varying smoke, outperforming prior methods.

Conclusion: SmokeSeer provides an effective solution for simultaneous 3D reconstruction and smoke removal, with open-source implementation and data available, addressing limitations of previous approaches.

Abstract: Smoke in real-world scenes can severely degrade image quality and hamper visibility. Recent image restoration methods either rely on data-driven priors that are susceptible to hallucinations, or are limited to static low-density smoke. We introduce SmokeSeer, a method for simultaneous 3D scene reconstruction and smoke removal from multi-view video sequences. Our method uses thermal and RGB images, leveraging the reduced scattering in thermal images to see through smoke. We build upon 3D Gaussian splatting to fuse information from the two image modalities, and decompose the scene into smoke and non-smoke components. Unlike prior work, SmokeSeer handles a broad range of smoke densities and adapts to temporally varying smoke. We validate our method on synthetic data and a new real-world smoke dataset with RGB and thermal images. We provide an open-source implementation and data on the project website.

Method: SSL4RL reformulates self-supervised learning objectives (like predicting image rotation or reconstructing masked patches) into dense, automatic reward signals for RL-based fine-tuning. This eliminates need for human preference data or unreliable AI evaluators.

Result: SSL4RL substantially improves performance on both vision-centric and vision-language reasoning benchmarks. Systematic ablations identify key factors: task difficulty, model scale, and semantic alignment with target domain. Framework also works for graph learning with significant gains.

Conclusion: SSL4RL establishes a versatile and effective paradigm for aligning multimodal models using verifiable, self-supervised objectives, offering new design principles for future work.

Abstract: Vision-language models (VLMs) have shown remarkable abilities by integrating large language models with visual inputs. However, they often fail to utilize visual evidence adequately, either depending on linguistic priors in vision-centric tasks or resorting to textual shortcuts during reasoning. Although reinforcement learning (RL) can align models with desired behaviors, its application to VLMs has been hindered by the lack of scalable and reliable reward mechanisms. To overcome this challenge, we propose SSL4RL, a novel framework that leverages self-supervised learning (SSL) tasks as a source of verifiable rewards for RL-based fine-tuning. Our approach reformulates SSL objectives-such as predicting image rotation or reconstructing masked patches-into dense, automatic reward signals, eliminating the need for human preference data or unreliable AI evaluators. Experiments show that SSL4RL substantially improves performance on both vision-centric and vision-language reasoning benchmarks. Furthermore, through systematic ablations, we identify key factors-such as task difficulty, model scale, and semantic alignment with the target domain-that influence the effectiveness of SSL4RL tasks, offering new design principles for future work. We also demonstrate the framework’s generality by applying it to graph learning, where it yields significant gains. SSL4RL establishes a versatile and effective paradigm for aligning multimodal models using verifiable, self-supervised objectives.

Method: UFC-MIL uses multiple-resolution images with a novel patch-wise loss that learns latent patterns and expresses uncertainty for classification. It features an attention-based architecture with neighbor patch aggregation, and calibrates aggregated predictions through patch-level uncertainty without requiring multiple iterative inferences.

Result: UFC-MIL shows superior performance in model calibration while achieving classification accuracy comparable to state-of-the-art methods on challenging public datasets.

Conclusion: UFC-MIL successfully addresses the calibration gap in MIL for histopathology, providing both accurate and trustworthy diagnostic predictions that mimic pathologist examination behaviors, with practical advantages for clinical deployment.

Abstract: With the increasing demand for histopathological specimen examination and diagnostic reporting, Multiple Instance Learning (MIL) has received heightened research focus as a viable solution for AI-centric diagnostic aid. Recently, to improve its performance and make it work more like a pathologist, several MIL approaches based on the use of multiple-resolution images have been proposed, delivering often higher performance than those that use single-resolution images. Despite impressive recent developments of multiple-resolution MIL, previous approaches only focus on improving performance, thereby lacking research on well-calibrated MIL that clinical experts can rely on for trustworthy diagnostic results. In this study, we propose Uncertainty-Focused Calibrated MIL (UFC-MIL), which more closely mimics the pathologists’ examination behaviors while providing calibrated diagnostic predictions, using multiple images with different resolutions. UFC-MIL includes a novel patch-wise loss that learns the latent patterns of instances and expresses their uncertainty for classification. Also, the attention-based architecture with a neighbor patch aggregation module collects features for the classifier. In addition, aggregated predictions are calibrated through patch-level uncertainty without requiring multiple iterative inferences, which is a key practical advantage. Against challenging public datasets, UFC-MIL shows superior performance in model calibration while achieving classification accuracy comparable to that of state-of-the-art methods.

Method: AGENet incorporates spatial relationships through edge-aware geodesic distance learning with three main components: (1) edge-aware geodesic distance learning module using iterative Fast Marching refinement, (2) adaptive prototype extraction with spatially-weighted aggregation, and (3) adaptive parameter learning that adjusts to different organ characteristics.

Result: Extensive experiments across diverse medical imaging datasets show improvements over state-of-the-art methods. The method reduces boundary errors while maintaining computational efficiency.

Conclusion: AGENet is highly suitable for clinical applications requiring precise segmentation with limited annotated data, leveraging predictable geometric patterns of medical structures through lightweight geometric modeling rather than complex neural networks.

Abstract: Medical image segmentation requires large annotated datasets, creating a significant bottleneck for clinical applications. While few-shot segmentation methods can learn from minimal examples, existing approaches demonstrate suboptimal performance in precise boundary delineation for medical images, particularly when anatomically similar regions appear without sufficient spatial context. We propose AGENet (Adaptive Geodesic Edge-aware Network), a novel framework that incorporates spatial relationships through edge-aware geodesic distance learning. Our key insight is that medical structures follow predictable geometric patterns that can guide prototype extraction even with limited training data. Unlike methods relying on complex architectural components or heavy neural networks, our approach leverages computationally lightweight geometric modeling. The framework combines three main components: (1) An edge-aware geodesic distance learning module that respects anatomical boundaries through iterative Fast Marching refinement, (2) adaptive prototype extraction that captures both global structure and local boundary details via spatially-weighted aggregation, and (3) adaptive parameter learning that automatically adjusts to different organ characteristics. Extensive experiments across diverse medical imaging datasets demonstrate improvements over state-of-the-art methods. Notably, our method reduces boundary errors compared to existing approaches while maintaining computational efficiency, making it highly suitable for clinical applications requiring precise segmentation with limited annotated data.

Method: Proposes View-aware Cross-modal Knowledge Distillation (ViCoKD) with: 1) Cross-modal adapter using cross-modal attention to exploit multi-modal correlations despite incomplete modalities, and 2) View-aware Consistency module that enforces prediction alignment for co-visible actions across views using human-detection masks and confidence-weighted Jensen-Shannon divergence.

Result: Experiments on MultiSensor-Home dataset show ViCoKD consistently outperforms competitive distillation methods across multiple backbones and environments, achieving significant gains and even surpassing the teacher model under limited conditions.

Conclusion: ViCoKD effectively addresses the challenges of partially overlapping multi-view setups with limited modalities and annotations by distilling knowledge from fully supervised teachers to constrained students, demonstrating practical value for real-world sensor systems.

Abstract: The widespread use of multi-sensor systems has increased research in multi-view action recognition. While existing approaches in multi-view setups with fully overlapping sensors benefit from consistent view coverage, partially overlapping settings where actions are visible in only a subset of views remain underexplored. This challenge becomes more severe in real-world scenarios, as many systems provide only limited input modalities and rely on sequence-level annotations instead of dense frame-level labels. In this study, we propose View-aware Cross-modal Knowledge Distillation (ViCoKD), a framework that distills knowledge from a fully supervised multi-modal teacher to a modality- and annotation-limited student. ViCoKD employs a cross-modal adapter with cross-modal attention, allowing the student to exploit multi-modal correlations while operating with incomplete modalities. Moreover, we propose a View-aware Consistency module to address view misalignment, where the same action may appear differently or only partially across viewpoints. It enforces prediction alignment when the action is co-visible across views, guided by human-detection masks and confidence-weighted Jensen-Shannon divergence between their predicted class distributions. Experiments on the real-world MultiSensor-Home dataset show that ViCoKD consistently outperforms competitive distillation methods across multiple backbones and environments, delivering significant gains and surpassing the teacher model under limited conditions.

Method: 1) Develop an Impartial World Model that learns to be honest about danger using Counterfactual Synthesis - a data synthesis pipeline generating plausible collisions and off-road events. 2) Integrate this model as an internal critic in a closed-loop RL framework where agents query it to “dream” of outcomes for candidate actions.

Result: The model significantly outperforms baselines in predicting failures on a new Risk Foreseeing Benchmark, and when used as a critic, enables substantial reduction in safety violations in challenging simulations.

Conclusion: Teaching world models to accurately dream of danger is critical for building truly safe and intelligent autonomous agents, addressing the fundamental optimistic bias problem in autonomous driving RL.

Abstract: End-to-end models for autonomous driving hold the promise of learning complex behaviors directly from sensor data, but face critical challenges in safety and handling long-tail events. Reinforcement Learning (RL) offers a promising path to overcome these limitations, yet its success in autonomous driving has been elusive. We identify a fundamental flaw hindering this progress: a deep seated optimistic bias in the world models used for RL. To address this, we introduce a framework for post-training policy refinement built around an Impartial World Model. Our primary contribution is to teach this model to be honest about danger. We achieve this with a novel data synthesis pipeline, Counterfactual Synthesis, which systematically generates a rich curriculum of plausible collisions and off-road events. This transforms the model from a passive scene completer into a veridical forecaster that remains faithful to the causal link between actions and outcomes. We then integrate this Impartial World Model into our closed-loop RL framework, where it serves as an internal critic. During refinement, the agent queries the critic to ``dream" of the outcomes for candidate actions. We demonstrate through extensive experiments, including on a new Risk Foreseeing Benchmark, that our model significantly outperforms baselines in predicting failures. Consequently, when used as a critic, it enables a substantial reduction in safety violations in challenging simulations, proving that teaching a model to dream of danger is a critical step towards building truly safe and intelligent autonomous agents.

Method: 1) Introduces HHOIs dataset and synthesizes data using image generative models. 2) Decomposes HHOIs into individual HOIs and HHIs. 3) Trains text-to-HOI and text-to-HHI models using score-based diffusion. 4) Develops unified generative framework integrating both models to synthesize complete HHOIs in a single sampling process. 5) Extends to multi-human settings beyond two individuals.

Result: The method generates realistic HHOIs conditioned on textual descriptions, outperforming previous approaches focused only on single-human HOIs. Successfully demonstrates multi-human motion generation involving objects as an application of the framework.

Conclusion: The paper presents a novel formulation for modeling Human-Human-Object Interactions, introduces a comprehensive dataset and generative framework, and demonstrates superior performance in generating realistic multi-person interactions with objects from text descriptions.

Abstract: The way humans interact with each other, including interpersonal distances, spatial configuration, and motion, varies significantly across different situations. To enable machines to understand such complex, context-dependent behaviors, it is essential to model multiple people in relation to the surrounding scene context. In this paper, we present a novel research problem to model the correlations between two people engaged in a shared interaction involving an object. We refer to this formulation as Human-Human-Object Interactions (HHOIs). To overcome the lack of dedicated datasets for HHOIs, we present a newly captured HHOIs dataset and a method to synthesize HHOI data by leveraging image generative models. As an intermediary, we obtain individual human-object interaction (HOIs) and human-human interaction (HHIs) from the HHOIs, and with these data, we train an text-to-HOI and text-to-HHI model using score-based diffusion model. Finally, we present a unified generative framework that integrates the two individual model, capable of synthesizing complete HHOIs in a single advanced sampling process. Our method extends HHOI generation to multi-human settings, enabling interactions involving more than two individuals. Experimental results show that our method generates realistic HHOIs conditioned on textual descriptions, outperforming previous approaches that focus only on single-human HOIs. Furthermore, we introduce multi-human motion generation involving objects as an application of our framework.

Method: Two novel mechanisms: 1) Motion Direction Decoupling (MDD) module decomposes temporal dynamics into signed polarity fields to encode both movement occurrence and trajectory direction, and 2) Residual-Driven Spatio-Temporal Refinement (R-STR) head uses Transformer-based coarse-to-fine refinement with factorized spatio-temporal contexts to recover occluded targets.

Result: Achieves new state-of-the-art F1-score of 0.9859 and accuracy of 0.9733 on TrackNetV2 dataset, significantly outperforming previous versions with only 3.7% increase in FLOPs compared to V4, maintaining real-time inference.

Conclusion: TrackNetV5 successfully addresses key limitations of previous versions through explicit motion direction encoding and residual refinement, delivering superior tracking precision for fast-moving small objects while preserving computational efficiency.

Abstract: The TrackNet series has established a strong baseline for fast-moving small object tracking in sports. However, existing iterations face significant limitations: V1-V3 struggle with occlusions due to a reliance on purely visual cues, while TrackNetV4, despite introducing motion inputs, suffers from directional ambiguity as its absolute difference method discards motion polarity. To overcome these bottlenecks, we propose TrackNetV5, a robust architecture integrating two novel mechanisms. First, to recover lost directional priors, we introduce the Motion Direction Decoupling (MDD) module. Unlike V4, MDD decomposes temporal dynamics into signed polarity fields, explicitly encoding both movement occurrence and trajectory direction. Second, we propose the Residual-Driven Spatio-Temporal Refinement (R-STR) head. Operating on a coarse-to-fine paradigm, this Transformer-based module leverages factorized spatio-temporal contexts to estimate a corrective residual, effectively recovering occluded targets. Extensive experiments on the TrackNetV2 dataset demonstrate that TrackNetV5 achieves a new state-of-the-art F1-score of 0.9859 and an accuracy of 0.9733, significantly outperforming previous versions. Notably, this performance leap is achieved with a marginal 3.7% increase in FLOPs compared to V4, maintaining real-time inference capabilities while delivering superior tracking precision.

Method: Built on MIT-Adobe FiveK dataset, DEAR incorporates pairwise human preference scores collected via large-scale crowdsourcing (25 evaluators per image pair, total 13,648 participants). The dataset enables Evaluation of Aesthetics of Rendering (EAR) as a new task.

Result: Created the first systematic dataset for image rendering aesthetics assessment grounded in subjective human preferences. Published subset of 100 images with markup on HuggingFace. The dataset captures nuanced, context-sensitive aesthetic preferences.

Conclusion: DEAR enables development of models that go beyond traditional distortion-based IQA, supporting style preference prediction, aesthetic benchmarking, and personalized aesthetic modeling for rendering aesthetics evaluation.

Abstract: Traditional Image Quality Assessment~(IQA) focuses on quantifying technical degradations such as noise, blur, or compression artifacts, using both full-reference and no-reference objective metrics. However, evaluation of rendering aesthetics, a growing domain relevant to photographic editing, content creation, and AI-generated imagery, remains underexplored due to the lack of datasets that reflect the inherently subjective nature of style preference. In this work, a novel benchmark dataset designed to model human aesthetic judgments of image rendering styles is introduced: the Dataset for Evaluating the Aesthetics of Rendering (DEAR). Built upon the MIT-Adobe FiveK dataset, DEAR incorporates pairwise human preference scores collected via large-scale crowdsourcing, with each image pair evaluated by 25 distinct human evaluators with a total of 13,648 of them participating overall. These annotations capture nuanced, context-sensitive aesthetic preferences, enabling the development and evaluation of models that go beyond traditional distortion-based IQA, focusing on a new task: Evaluation of Aesthetics of Rendering (EAR). The data collection pipeline is described, human voting patterns are analyzed, and multiple use cases are outlined, including style preference prediction, aesthetic benchmarking, and personalized aesthetic modeling. To the best of the authors’ knowledge, DEAR is the first dataset to systematically address image aesthetics of rendering assessment grounded in subjective human preferences. A subset of 100 images with markup for them is published on HuggingFace (huggingface.co/datasets/vsevolodpl/DEAR).

Method: Cross-Modal Alignment Consistency (CMAC-MMD) training framework that standardizes diagnostic certainty across intersectional patient subgroups without requiring sensitive demographic data during clinical inference.

Result: In dermatology: reduced intersectional missed diagnosis gap (ΔTPR) from 0.50 to 0.26 while improving AUC from 0.94 to 0.97. In glaucoma screening: reduced ΔTPR from 0.41 to 0.31, achieving better AUC of 0.72 vs 0.71 baseline.

Conclusion: Establishes a scalable framework for developing high-stakes clinical decision support systems that are both accurate and equitable across diverse patient subgroups without increasing privacy risks.

Abstract: Medical artificial intelligence (AI) systems, particularly multimodal vision-language models (VLM), often exhibit intersectional biases where models are systematically less confident in diagnosing marginalised patient subgroups. Such bias can lead to higher rates of inaccurate and missed diagnoses due to demographically skewed data and divergent distributions of diagnostic certainty. Current fairness interventions frequently fail to address these gaps or compromise overall diagnostic performance to achieve statistical parity among the subgroups. In this study, we developed Cross-Modal Alignment Consistency (CMAC-MMD), a training framework that standardises diagnostic certainty across intersectional patient subgroups. Unlike traditional debiasing methods, this approach equalises the model’s decision confidence without requiring sensitive demographic data during clinical inference. We evaluated this approach using 10,015 skin lesion images (HAM10000) with external validation on 12,000 images (BCN20000), and 10,000 fundus images for glaucoma detection (Harvard-FairVLMed), stratifying performance by intersectional age, gender, and race attributes. In the dermatology cohort, the proposed method reduced the overall intersectional missed diagnosis gap (difference in True Positive Rate, $Δ$TPR) from 0.50 to 0.26 while improving the overall Area Under the Curve (AUC) from 0.94 to 0.97 compared to standard training. Similarly, for glaucoma screening, the method reduced $Δ$TPR from 0.41 to 0.31, achieving a better AUC of 0.72 (vs. 0.71 baseline). This establishes a scalable framework for developing high-stakes clinical decision support systems that are both accurate and can perform equitably across diverse patient subgroups, ensuring reliable performance without increasing privacy risks.

Method: Proposes DiffusionVL, a method to translate any powerful AR model into a diffusion-based VLM through simple fine-tuning. Introduces block-decoding design that supports arbitrary-length generation and KV cache reuse for faster inference.

Result: Despite using less than 5% of training data compared to prior methods, DiffusionVL achieves 34.4% gain on MMMU-Pro (vision) and 37.5% gain on MME (Cog.) benchmarks, with 2x inference speedup. Performance is competitive with LLaVA-style visual-instruction-tuning.

Conclusion: Paradigm shift from AR to diffusion is effective and feasible. DiffusionVL demonstrates that powerful AR models can be successfully converted to diffusion-based VLMs with improved performance, efficiency, and reduced data requirements.

Abstract: In recent multimodal research, the diffusion paradigm has emerged as a promising alternative to the autoregressive paradigm (AR), owing to its unique decoding advantages. However, due to the capability limitations of the base diffusion language model, the performance of the diffusion vision language model (dVLM) still lags significantly behind that of mainstream models. This leads to a simple yet fundamental question: Is it possible to construct dVLMs based on existing powerful AR models? In response, we propose DiffusionVL, a dVLM family that could be translated from any powerful AR models. Through simple fine-tuning, we successfully adapt AR pre-trained models into the diffusion paradigm. This approach yields two key observations: (1) The paradigm shift from AR-based multimodal models to diffusion is remarkably effective. (2) Direct conversion of an AR language model to a dVLM is also feasible, achieving performance competitive with LLaVA-style visual-instruction-tuning. Further, we introduce a block-decoding design into dVLMs that supports arbitrary-length generation and KV cache reuse, achieving a significant inference speedup. We conduct a large number of experiments. Despite training with less than 5% of the data required by prior methods, DiffusionVL achieves a comprehensive performance improvement-a 34.4% gain on the MMMU-Pro (vision) bench and 37.5% gain on the MME (Cog.) bench-alongside a 2x inference speedup. The model and code are released at https://github.com/hustvl/DiffusionVL.

Method: Developed CBAM-VGG16 model that integrates Convolution Block Attention Module (CBAM) at each convolutional stage. Trained on five diverse plant disease datasets and evaluated using attention maps, Grad-CAM, Grad-CAM++, and Layer-wise Relevance Propagation (LRP) for interpretability.

Result: Achieved high accuracy up to 98.87%, outperforming recent techniques. Demonstrated robust generalization across datasets and provided transparent interpretability through attention visualization.

Conclusion: The study advances explainable AI in agricultural diagnostics by offering a transparent and reliable system for smart farming, with code publicly available on GitHub.

Abstract: Plant diseases pose a significant threat to global food security, necessitating accurate and interpretable disease detection methods. This study introduces an interpretable attention-guided Convolutional Neural Network (CNN), CBAM-VGG16, for plant leaf disease detection. By integrating Convolution Block Attention Module (CBAM) at each convolutional stage, the model enhances feature extraction and disease localization. Trained on five diverse plant disease datasets, our approach outperforms recent techniques, achieving high accuracy (up to 98.87%) and demonstrating robust generalization. Here, we show the effectiveness of our method through comprehensive evaluation and interpretability analysis using CBAM attention maps, Grad-CAM, Grad-CAM++, and Layer-wise Relevance Propagation (LRP). This study advances the application of explainable AI in agricultural diagnostics, offering a transparent and reliable system for smart farming. The code of our proposed work is available at https://github.com/BS0111/PlantAttentionCBAM.

Method: FALCON-SFOD has two components: 1) SPAR (Spatial Prior-Aware Regularization) uses vision foundation models (OV-SAM) to generate class-agnostic binary masks that regularize the detector’s feature space toward object regions. 2) IRPL (Imbalance-aware Noise Robust Pseudo-Labeling) promotes balanced and noise-tolerant learning under severe foreground-background imbalance.

Result: The framework achieves competitive performance across SFOD benchmarks, with theoretical analysis showing tighter localization and classification error bounds.

Conclusion: Strengthening object-focused feature representations through foundation model alignment and noise-robust pseudo-labeling is crucial for effective source-free object detection under domain shift.

Abstract: Current state-of-the-art approaches in Source-Free Object Detection (SFOD) typically rely on Mean-Teacher self-labeling. However, domain shift often reduces the detector’s ability to maintain strong object-focused representations, causing high-confidence activations over background clutter. This weak object focus results in unreliable pseudo-labels from the detection head. While prior works mainly refine these pseudo-labels, they overlook the underlying need to strengthen the feature space itself. We propose FALCON-SFOD (Foundation-Aligned Learning with Clutter suppression and Noise robustness), a framework designed to enhance object-focused adaptation under domain shift. It consists of two complementary components. SPAR (Spatial Prior-Aware Regularization) leverages the generalization strength of vision foundation models to regularize the detector’s feature space. Using class-agnostic binary masks derived from OV-SAM, SPAR promotes structured and foreground-focused activations by guiding the network toward object regions. IRPL (Imbalance-aware Noise Robust Pseudo-Labeling) complements SPAR by promoting balanced and noise-tolerant learning under severe foreground-background imbalance. Guided by a theoretical analysis that connects these designs to tighter localization and classification error bounds, FALCON-SFOD achieves competitive performance across SFOD benchmarks.

Method: Proposes SVLP-IL framework with Multi-Aspect Prompting (MAP) to isolate domain dependencies, enhance distribution-shift sensitivity, and mitigate forgetting by exploiting universal and domain-specific cues. Uses Selective Elastic Weight Consolidation (SEWC) to preserve critical weights from previous tasks while allowing flexibility for new adaptations.

Result: Comprehensive experiments across multiple PAD benchmarks show SVLP-IL significantly reduces catastrophic forgetting and enhances performance on unseen domains.

Conclusion: SVLP-IL offers a privacy-compliant, practical solution for robust lifelong PAD deployment in rehearsal-free incremental learning settings.

Abstract: Face Presentation Attack Detection (PAD) demands incremental learning (IL) to combat evolving spoofing tactics and domains. Privacy regulations, however, forbid retaining past data, necessitating rehearsal-free IL (RF-IL). Vision-Language Pre-trained (VLP) models, with their prompt-tunable cross-modal representations, enable efficient adaptation to new spoofing styles and domains. Capitalizing on this strength, we propose \textbf{SVLP-IL}, a VLP-based RF-IL framework that balances stability and plasticity via \textit{Multi-Aspect Prompting} (MAP) and \textit{Selective Elastic Weight Consolidation} (SEWC). MAP isolates domain dependencies, enhances distribution-shift sensitivity, and mitigates forgetting by jointly exploiting universal and domain-specific cues. SEWC selectively preserves critical weights from previous tasks, retaining essential knowledge while allowing flexibility for new adaptations. Comprehensive experiments across multiple PAD benchmarks show that SVLP-IL significantly reduces catastrophic forgetting and enhances performance on unseen domains. SVLP-IL offers a privacy-compliant, practical solution for robust lifelong PAD deployment in RF-IL settings.

Method: Two novel methods: 1) Anatomical Similarity Curriculum Learning - progressive learning strategy controlling question difficulty via similarity of answer choices, enabling incremental mastery of complex problems. 2) Group Diversity Question Augmentation - question augmentation to expand model’s search space for difficult queries, mitigating uniform response tendencies.

Result: Comprehensive experiments on SGG-VQA and OmniMedVQA benchmarks show significant improvement across both benchmarks, demonstrating effectiveness in enhancing medical reasoning capabilities of MLLMs.

Conclusion: The proposed methods effectively address GRPO’s weaknesses in medical anatomy reasoning by enabling progressive learning and expanding reasoning diversity, leading to improved performance on medical VQA benchmarks.

Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive progress in natural image reasoning, yet their potential in medical imaging remains underexplored, especially in clinical anatomical surgical images. Anatomy understanding tasks demand precise understanding and clinically coherent answers, which are difficult to achieve due to the complexity of medical data and the scarcity of high-quality expert annotations. These challenges limit the effectiveness of conventional Supervised Fine-Tuning (SFT) strategies. While recent work has demonstrated that Group Relative Policy Optimization (GRPO) can enhance reasoning in MLLMs without relying on large amounts of data, we find two weaknesses that hinder GRPO’s reasoning performance in anatomy recognition: 1) knowledge cannot be effectively shared between different anatomical structures, resulting in uneven information gain and preventing the model from converging, and 2) the model quickly converges to a single reasoning path, suppressing the exploration of diverse strategies. To overcome these challenges, we propose two novel methods. First, we implement a progressive learning strategy called Anatomical Similarity Curriculum Learning by controlling question difficulty via the similarity of answer choices, enabling the model to master complex problems incrementally. Second, we utilize question augmentation referred to as Group Diversity Question Augmentation to expand the model’s search space for difficult queries, mitigating the tendency to produce uniform responses. Comprehensive experiments on the SGG-VQA and OmniMedVQA benchmarks show our method achieves a significant improvement across the two benchmarks, demonstrating its effectiveness in enhancing the medical reasoning capabilities of MLLMs. The code can be found in https://github.com/tomato996/Anatomy-R1

Method: Uses video diffusion models to generate perceptually accurate focal stacks from single defocused images. The focal stack is represented as a video sequence. The authors also release a large-scale focal stack dataset acquired under diverse real-world smartphone conditions.

Result: The method consistently outperforms existing approaches in both perceptual quality and robustness across challenging scenarios. It enables interactive refocusing and unlocks various downstream applications.

Conclusion: This approach paves the way for more advanced focus-editing capabilities in everyday photography. The released code and dataset support future research in this area.

Abstract: Focus is a cornerstone of photography, yet autofocus systems often fail to capture the intended subject, and users frequently wish to adjust focus after capture. We introduce a novel method for realistic post-capture refocusing using video diffusion models. From a single defocused image, our approach generates a perceptually accurate focal stack, represented as a video sequence, enabling interactive refocusing and unlocking a range of downstream applications. We release a large-scale focal stack dataset acquired under diverse real-world smartphone conditions to support this work and future research. Our method consistently outperforms existing approaches in both perceptual quality and robustness across challenging scenarios, paving the way for more advanced focus-editing capabilities in everyday photography. Code and data are available at https://learn2refocus.github.io

Method: FedPOD defines round-wise tasks, includes participants excluded as outliers, eliminates dependency on previous rounds’ learning information, applies validation loss calculation at each round, and is designed to be compatible with Kubernetes auto-scaling using POD units.

Result: Achieved first place in 2024 FeTS Challenge with Dice scores of 0.78 (WT), 0.71 (ET), 0.72 (TC) on average, and projected convergence score of 0.74, comparable to FedPIDAvg performance.

Conclusion: FedPOD demonstrates potential to enhance federated learning by improving efficiency, flexibility, and performance metrics while being compatible with Kubernetes auto-scaling for flexible deployment.

Abstract: This paper proposes FedPOD, which ranked first in the 2024 Federated Tumor Segmentation (FeTS) Challenge, for optimizing learning efficiency and communication cost in federated learning among multiple clients. Inspired by FedPIDAvg, we define a round-wise task for FedPOD to enhance training efficiency. FedPIDAvg achieved performance improvement by incorporating the training loss reduction for prediction entropy as weights using differential terms. Furthermore, by modeling data distribution with a Poisson distribution and using a PID controller, it reduced communication costs even in skewed data distribution. However, excluding participants classified as outliers based on the Poisson distribution can limit data utilization. Additionally, PID controller requires the same participants to be maintained throughout the federated learning process as it uses previous rounds’ learning information in the current round. In our approach, FedPOD addresses these issues by including participants excluded as outliers, eliminating dependency on previous rounds’ learning information, and applying a method for calculating validation loss at each round. In this challenge, FedPOD presents comparable performance to FedPIDAvg in metrics of Dice score, 0.78, 0.71 and 0.72 for WT, ET and TC in average, and projected convergence score, 0.74 in average. Furthermore, the concept of FedPOD draws inspiration from Kubernetes’ smallest computing unit, POD, designed to be compatible with Kubernetes auto-scaling. Extending round-wise tasks of FedPOD to POD units allows flexible design by applying scale-out similar to Kubernetes’ auto-scaling. This work demonstrated the potentials of FedPOD to enhance federated learning by improving efficiency, flexibility, and performance in metrics.

Method: Two-stage diffusion process: 1) First diffusion model generates compact semantic video features for global planning, 2) Second diffusion model generates VAE latents conditioned on semantic features to add high-frequency details.

Result: SemanticGen achieves faster convergence than VAE latent space methods, is computationally efficient for long videos, and outperforms state-of-the-art approaches in video quality.

Conclusion: Generating videos in semantic space with a two-stage diffusion approach is more efficient and effective than direct VAE latent modeling, enabling better long video generation with faster convergence.

Abstract: State-of-the-art video generative models typically learn the distribution of video latents in the VAE space and map them to pixels using a VAE decoder. While this approach can generate high-quality videos, it suffers from slow convergence and is computationally expensive when generating long videos. In this paper, we introduce SemanticGen, a novel solution to address these limitations by generating videos in the semantic space. Our main insight is that, due to the inherent redundancy in videos, the generation process should begin in a compact, high-level semantic space for global planning, followed by the addition of high-frequency details, rather than directly modeling a vast set of low-level video tokens using bi-directional attention. SemanticGen adopts a two-stage generation process. In the first stage, a diffusion model generates compact semantic video features, which define the global layout of the video. In the second stage, another diffusion model generates VAE latents conditioned on these semantic features to produce the final output. We observe that generation in the semantic space leads to faster convergence compared to the VAE latent space. Our method is also effective and computationally efficient when extended to long video generation. Extensive experiments demonstrate that SemanticGen produces high-quality videos and outperforms state-of-the-art approaches and strong baselines.

Method: Combines 1-bit quantized Llama-3.2-1B LLM with Deep Q-Network (DQN) reinforcement learning deployed on Raspberry Pi hardware. Uses multi-objective RL to learn optimal lighting policies from user feedback, balancing energy consumption, comfort, and circadian alignment. Integrates with Google Home/IFTTT for natural language commands and learns from implicit feedback through manual overrides.

Result: Achieves 71.4 times energy reduction compared to full-precision models, 32% energy savings over rule-based systems, inference latency under 200ms on Raspberry Pi 4, 95% user satisfaction, and 5.07 times speedup over 2-bit alternatives on ARM processors while maintaining 92% task accuracy.

Conclusion: Establishes a practical framework for deploying adaptive AI on resource-constrained IoT devices, enabling intelligent home automation without cloud dependencies through efficient 1-bit quantization and reinforcement learning integration.

Abstract: Smart home lighting systems consume 15-20% of residential energy but lack adaptive intelligence to optimize for user comfort and energy efficiency simultaneously. We present BitRL-Light, a novel framework combining 1-bit quantized Large Language Models (LLMs) with Deep Q-Network (DQN) reinforcement learning for real-time smart home lighting control on edge devices. Our approach deploys a 1-bit quantized Llama-3.2-1B model on Raspberry Pi hardware, achieving 71.4 times energy reduction compared to full-precision models while maintaining intelligent control capabilities. Through multi-objective reinforcement learning, BitRL-Light learns optimal lighting policies from user feedback, balancing energy consumption, comfort, and circadian alignment. Experimental results demonstrate 32% energy savings compared to rule-based systems, with inference latency under 200ms on Raspberry Pi 4 and 95% user satisfaction. The system processes natural language commands via Google Home/IFTTT integration and learns from implicit feedback through manual overrides. Our comparative analysis shows 1-bit models achieve 5.07 times speedup over 2-bit alternatives on ARM processors while maintaining 92% task accuracy. This work establishes a practical framework for deploying adaptive AI on resource-constrained IoT devices, enabling intelligent home automation without cloud dependencies.

Method: Integrates classical MARL with quantum-inspired optimization using variational quantum circuits (VQCs) and QAOA for combinatorial optimization. Incorporates Bayesian inference, Gaussian processes, and variational inference for probabilistic modeling. Uses CTDE paradigm with shared memory and local view grids to enhance observability among 10 cooperative UAVs.

Result: The framework improves sample efficiency, accelerates convergence, and enhances coverage performance while maintaining robustness. QI-MARL achieves superior balance between exploration and exploitation compared to PPO and DDPG baselines, as shown through scalability tests, sensitivity analysis, and convergence analyses.

Conclusion: Quantum-inspired optimization techniques combined with probabilistic modeling can effectively enhance MARL performance for complex cooperative tasks like UAV-assisted 6G network deployment, offering better exploration-exploitation balance and practical advantages over classical approaches.

Abstract: This study introduces a quantum inspired framework for optimizing the exploration exploitation tradeoff in multiagent reinforcement learning, applied to UAVassisted 6G network deployment. We consider a cooperative scenario where ten intelligent UAVs autonomously coordinate to maximize signal coverage and support efficient network expansion under partial observability and dynamic conditions. The proposed approach integrates classical MARL algorithms with quantum-inspired optimization techniques, leveraging variational quantum circuits VQCs as the core structure and employing the Quantum Approximate Optimization Algorithm QAOA as a representative VQC based method for combinatorial optimization. Complementary probabilistic modeling is incorporated through Bayesian inference, Gaussian processes, and variational inference to capture latent environmental dynamics. A centralized training with decentralized execution CTDE paradigm is adopted, where shared memory and local view grids enhance local observability among agents. Comprehensive experiments including scalability tests, sensitivity analysis, and comparisons with PPO and DDPG baselines demonstrate that the proposed framework improves sample efficiency, accelerates convergence, and enhances coverage performance while maintaining robustness. Radar chart and convergence analyses further show that QI MARL achieves a superior balance between exploration and exploitation compared to classical methods. All implementation code and supplementary materials are publicly available on GitHub to ensure reproducibility.

Method: Introduces a multimodal knowledge graph-based RAG system that incorporates visual cues into three key stages: (1) knowledge graph construction, (2) retrieval phase, and (3) answer generation process. This enables cross-modal reasoning by integrating textual, visual, and spatial cues into structured hierarchical concepts.

Result: Experimental results across both global and fine-grained question answering tasks show that the approach consistently outperforms existing RAG-based approaches on both textual and multimodal corpora.

Conclusion: The proposed multimodal knowledge graph-based RAG system successfully addresses limitations of existing approaches by enabling cross-modal reasoning through visual cue integration, leading to improved content understanding and superior performance on QA tasks.

Abstract: Retrieval-augmented generation (RAG) enables large language models (LLMs) to dynamically access external information, which is powerful for answering questions over previously unseen documents. Nonetheless, they struggle with high-level conceptual understanding and holistic comprehension due to limited context windows, which constrain their ability to perform deep reasoning over long-form, domain-specific content such as full-length books. To solve this problem, knowledge graphs (KGs) have been leveraged to provide entity-centric structure and hierarchical summaries, offering more structured support for reasoning. However, existing KG-based RAG solutions remain restricted to text-only inputs and fail to leverage the complementary insights provided by other modalities such as vision. On the other hand, reasoning from visual documents requires textual, visual, and spatial cues into structured, hierarchical concepts. To address this issue, we introduce a multimodal knowledge graph-based RAG that enables cross-modal reasoning for better content understanding. Our method incorporates visual cues into the construction of knowledge graphs, the retrieval phase, and the answer generation process. Experimental results across both global and fine-grained question answering tasks show that our approach consistently outperforms existing RAG-based approaches on both textual and multimodal corpora.

Method: Organized as an international conference with double-blind peer review process for paper selection, featuring proceedings published in cooperation with IEICE Proceedings Series, with selected papers recommended for further publication in IEICE Transactions on Information and Systems after additional review.

Result: Successful organization of KICSS 2025 conference with proceedings containing peer-reviewed papers, establishing a platform for knowledge sharing and collaboration among researchers from multiple disciplines, with quality papers recommended for journal publication.

Conclusion: The KICSS 2025 conference successfully served as an important venue for advancing research in knowledge, information, and creativity support systems, demonstrating the value of multidisciplinary collaboration and rigorous peer review processes in producing high-quality research contributions.

Abstract: This volume presents the proceedings of the 20th International Conference on Knowledge, Information and Creativity Support Systems (KICSS 2025), held in Nagaoka, Japan, on December 3-5, 2025. The conference, organized in cooperation with the IEICE Proceedings Series, provides a multidisciplinary forum for researchers in artificial intelligence, knowledge engineering, human-computer interaction, and creativity support systems. The proceedings include peer-reviewed papers accepted through a double-blind review process. Selected papers have been recommended for publication in IEICE Transactions on Information and Systems after an additional peer-review process.

Method: Microprobe combines strategic prompt diversity across five key reliability dimensions with advanced uncertainty quantification and adaptive weighting. It uses only 100 strategically selected probe examples to efficiently detect potential failure modes.

Result: Achieves 23.5% higher composite reliability scores compared to random sampling baselines with exceptional statistical significance (p < 0.001, Cohen’s d = 1.21). Expert validation rates the approach 4.14/5.0 vs 3.14/5.0 for random selection. Completes assessment with 99.9% statistical power, 90% cost reduction, and maintains 95% of traditional method coverage.

Conclusion: Microprobe addresses a critical gap in efficient model evaluation for responsible AI deployment by enabling comprehensive reliability assessment with dramatically reduced computational requirements while maintaining high statistical power and coverage.

Abstract: Foundation model reliability assessment typically requires thousands of evaluation examples, making it computationally expensive and time-consuming for real-world deployment. We introduce microprobe, a novel approach that achieves comprehensive reliability assessment using only 100 strategically selected probe examples. Our method combines strategic prompt diversity across five key reliability dimensions with advanced uncertainty quantification and adaptive weighting to efficiently detect potential failure modes. Through extensive empirical evaluation on multiple language models (GPT-2 variants, GPT-2 Medium, GPT-2 Large) and cross-domain validation (healthcare, finance, legal), we demonstrate that microprobe achieves 23.5% higher composite reliability scores compared to random sampling baselines, with exceptional statistical significance (p < 0.001, Cohen’s d = 1.21). Expert validation by three AI safety researchers confirms the effectiveness of our strategic selection, rating our approach 4.14/5.0 versus 3.14/5.0 for random selection. microprobe completes reliability assessment with 99.9% statistical power while representing a 90% reduction in assessment cost and maintaining 95% of traditional method coverage. Our approach addresses a critical gap in efficient model evaluation for responsible AI deployment.

Method: Introduces multi-agent with multi-persona debators to generate reflections, creating more diverse and effective feedback than single-LLM reflection.

Result: Achieves 47% EM on HotPot QA and 82.7% on HumanEval, surpassing single-LLM reflection performance on both reasoning and programming tasks.

Conclusion: Multi-agent multi-persona debating approach effectively addresses reflection degeneration and improves LLM reasoning performance through diverse feedback generation.

Abstract: LLMs have shown the capacity to improve their performance on reasoning tasks through reflecting on their mistakes, and acting with these reflections in mind. However, continual reflections of the same LLM onto itself exhibit degeneration of thought, where the LLM continues to repeat the same errors again and again even with the knowledge that its wrong. To address this problem, we instead introduce multi-agent with multi-persona debators as the method to generate reflections. Through out extensive experimentation, we’ve found that the leads to better diversity of in the reflections generated by the llm agent. We demonstrate an accuracy of 47% EM HotPot QA (question answering) and 82.7% on HumanEval (programming), both performances surpassing reflection with a single llm.

Method: Built on Alibaba Qwen-3 model with ~500GB of structured medical knowledge, using hybrid enhanced pre-training and retrieval-enhanced generation (RAG) approach for accurate symptom understanding and analysis.

Result: The model achieves efficient 3-5 round interactions for symptom analysis and diagnostic suggestions, and outperforms GPT-4 on comprehensive medical exams.

Conclusion: Erkang-Diagnosis-1.1 successfully creates an intelligent health companion that enhances primary healthcare accessibility and demonstrates superior performance to existing models in medical evaluation.

Abstract: This report provides a detailed introduction to Erkang-Diagnosis-1.1 model, our AI healthcare consulting assistant developed using Alibaba Qwen-3 model. The Erkang model integrates approximately 500GB of high-quality structured medical knowledge, employing a hybrid approach combining enhanced pre-training and retrieval-enhanced generation to create a secure, reliable, and professional AI health advisor. Through 3-5 efficient interaction rounds, Erkang Diagnosis can accurately understand user symptoms, conduct preliminary analysis, and provide valuable diagnostic suggestions and health guidance. Designed to become users intelligent health companions, it empowers primary healthcare and health management. To validate, Erkang-Diagnosis-1.1 leads GPT-4 in terms of comprehensive medical exams.

Method: Researchers use token-level log-probability thresholds to truncate reasoning chains at early, mid, and late stages from baseline models (Gemma-3-4B-IT and LLaMA-3.1-70B-Instruct). They conduct continuation experiments with smaller models (Gemma-3-1B-IT and LLaMA-3.1-8B-Instruct) to test intra-family and cross-family behaviors, using a Process Reward Model (PRM) evaluation pipeline.

Result: Hybrid reasoning chains often preserve and sometimes even improve final accuracy and logical structure. Interchangeability emerges as a behavioral property of reasoning models, with reasoning remaining coherent under model substitution.

Conclusion: Reasoning interchangeability offers insights into new paradigms for reliable modular reasoning in collaborative AI systems, suggesting that reasoning scaffolds can be transferred across models while maintaining coherence and accuracy.

Abstract: Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of large language models (LLMs). While prior work focuses on improving model performance through internal reasoning strategies, little is known about the interchangeability of reasoning across different models. In this work, we explore whether a partially completed reasoning chain from one model can be reliably continued by another model, either within the same model family or across families. We achieve this by assessing the sufficiency of intermediate reasoning traces as transferable scaffolds for logical coherence and final answer accuracy. We interpret this interchangeability as a means of examining inference-time trustworthiness, probing whether reasoning remains both coherent and reliable under model substitution. Using token-level log-probability thresholds to truncate reasoning at early, mid, and late stages from our baseline models, Gemma-3-4B-IT and LLaMA-3.1-70B-Instruct, we conduct continuation experiments with Gemma-3-1B-IT and LLaMA-3.1-8B-Instruct to test intra-family and cross-family behaviors. Our evaluation pipeline leverages truncation thresholds with a Process Reward Model (PRM), providing a reproducible framework for assessing reasoning stability via model interchange. Evaluations with a PRM reveal that hybrid reasoning chains often preserve, and in some cases even improve, final accuracy and logical structure. Our findings point towards interchangeability as an emerging behavioral property of reasoning models, offering insights into new paradigms for reliable modular reasoning in collaborative AI systems.

Method: Proposes a unified architecture combining LangChain-based multi-agent systems with permissioned blockchain (Hyperledger Fabric). The framework links perception-conceptualization-action cycles to blockchain governance layers that verify inputs, evaluate actions, and document outcomes. Uses MCP-integrated action executors and LangChain agents.

Result: Experiments in smart inventory management, traffic-signal control, and healthcare monitoring show blockchain security verification effectively prevents unauthorized practices, provides full decision-making traceability, and maintains operational latency within acceptable ranges.

Conclusion: The framework offers a universal approach for implementing high-impact agentic AI applications that balance autonomy with responsibility through immutable auditing and policy enforcement.

Abstract: The application of agentic AI systems in autonomous decision-making is growing in the areas of healthcare, smart cities, digital forensics, and supply chain management. Even though these systems are flexible and offer real-time reasoning, they also raise concerns of trust and oversight, and integrity of the information and activities upon which they are founded. The paper suggests a single architecture model comprising of LangChain-based multi-agent system with a permissioned blockchain to guarantee constant monitoring, policy enforcement, and immutable auditability of agentic action. The framework relates the perception conceptualization-action cycle to a blockchain layer of governance that verifies the inputs, evaluates recommended actions, and documents the outcomes of the execution. A Hyperledger Fabric-based system, action executors MCP-integrated, and LangChain agent are introduced and experiments of smart inventory management, traffic-signal control, and healthcare monitoring are done. The results suggest that blockchain-security verification is efficient in preventing unauthorized practices, offers traceability throughout the whole decision-making process, and maintains operational latency within reasonable ranges. The suggested framework provides a universal system of implementing high-impact agentic AI applications that are autonomous yet responsible.

Method: AAT uses decentralized identity (DID) and verifiable credentials (VC) to establish trusted AI entities. It models AI entities as nodes in a dynamic interaction graph with edges representing time-specific behavioral trajectories. A risk diffusion algorithm traces risky behavior origins and propagates warnings across involved entities. The system records inter-entity interaction trajectories on-chain for cross-system supervision.

Result: The system’s performance was evaluated using blockchain Transactions Per Second (TPS) metrics, demonstrating feasibility and stability under large-scale interaction recording. AAT provides a scalable and verifiable solution for AI auditing.

Conclusion: AAT offers a practical framework for AI usage traffic recording and governance that enables risk management, responsibility attribution, and cross-system auditing in complex multi-agent environments through blockchain technology.

Abstract: The rapid expansion of AI-driven applications powered by large language models has led to a surge in AI interaction data, raising urgent challenges in security, accountability, and risk traceability. This paper presents AiAuditTrack (AAT), a blockchain-based framework for AI usage traffic recording and governance. AAT leverages decentralized identity (DID) and verifiable credentials (VC) to establish trusted and identifiable AI entities, and records inter-entity interaction trajectories on-chain to enable cross-system supervision and auditing. AI entities are modeled as nodes in a dynamic interaction graph, where edges represent time-specific behavioral trajectories. Based on this model, a risk diffusion algorithm is proposed to trace the origin of risky behaviors and propagate early warnings across involved entities. System performance is evaluated using blockchain Transactions Per Second (TPS) metrics, demonstrating the feasibility and stability of AAT under large-scale interaction recording. AAT provides a scalable and verifiable solution for AI auditing, risk management, and responsibility attribution in complex multi-agent environments.

Method: Price-aware agentic AI system with modular multi-agent architecture (budgeting, nutrition, price monitoring, and health personalization agents). Uses shared knowledge base and substitution graphs to maintain nutritional quality at minimum cost, incorporating household income, fixed expenditures, medical status, and real-time food prices.

Result: Simulations with Saudi household case study show 12-18% cost reduction vs static weekly menu, over 95% nutrient adequacy, and robust performance with 20-30% price changes.

Conclusion: The framework successfully combines affordability with nutritional adequacy, providing a viable approach for sustainable and equitable diet planning aligned with Sustainable Development Goals on Zero Hunger and Good Health.

Abstract: The issue of limited household budgets and nutritional demands continues to be a challenge especially in the middle-income environment where food prices fluctuate. This paper introduces a price aware agentic AI system, which combines personal finance management with diet optimization. With household income and fixed expenditures, medical and well-being status, as well as real-time food costs, the system creates nutritionally sufficient meals plans at comparatively reasonable prices that automatically adjust to market changes. The framework is implemented in a modular multi-agent architecture, which has specific agents (budgeting, nutrition, price monitoring, and health personalization). These agents share the knowledge base and use the substitution graph to ensure that the nutritional quality is maintained at a minimum cost. Simulations with a representative Saudi household case study show a steady 12-18% reduction in costs relative to a static weekly menu, nutrient adequacy of over 95% and high performance with price changes of 20-30%. The findings indicate that the framework can locally combine affordability with nutritional adequacy and provide a viable avenue of capacity-building towards sustainable and fair diet planning in line with Sustainable Development Goals on Zero Hunger and Good Health.

Method: Mixture of Attention Schemes (MoAS) with learned router that dynamically selects optimal attention scheme (MHA, GQA, or MQA) for each token, rather than static averaging.

Result: Dynamic routing (val loss 2.3074) outperforms static mixture (2.3093) on WikiText-2, achieving performance competitive with MHA baseline while offering conditional compute efficiency.

Conclusion: MoAS enables dynamic attention scheme selection per token, providing better performance than static approaches and potential inference efficiency gains while maintaining model quality.

Abstract: The choice of attention mechanism in Transformer models involves a critical trade-off between modeling quality and inference efficiency. Multi-Head Attention (MHA) offers the best quality but suffers from large Key-Value (KV) cache memory requirements during inference. Multi-Query Attention (MQA) and Grouped-Query Attention (GQA) reduce memory usage but often at the cost of model performance. In this work, we propose Mixture of Attention Schemes (MoAS), a novel architecture that dynamically selects the optimal attention scheme (MHA, GQA, or MQA) for each token via a learned router. We demonstrate that dynamic routing performs better than static averaging of schemes and achieves performance competitive with the MHA baseline while offering potential for conditional compute efficiency. Experimental results on WikiText-2 show that dynamic routing (val loss 2.3074) outperforms a static mixture (2.3093), validating the effectiveness of the proposed method. Our code is available at https://github.com/Esmail-ibraheem/Mixture-of-Attention-Schemes-MoAS.

Method: Proposes Memory Bear system with human-like memory architecture based on cognitive science principles, integrating multimodal information perception, dynamic memory maintenance, and adaptive cognitive services for full-chain reconstruction of LLM memory mechanisms.

Result: Outperforms existing solutions (Mem0, MemGPT, Graphiti) across key metrics including accuracy, token efficiency, and response latency; improves knowledge fidelity, retrieval efficiency, reduces hallucination rates, and enhances contextual adaptability and reasoning.

Conclusion: Memory Bear represents a crucial advancement in moving AI from “memory” to “cognition” by addressing fundamental LLM memory limitations through cognitive science-inspired architecture, with demonstrated effectiveness across healthcare, enterprise, and education domains.

Abstract: Large language models (LLMs) face inherent limitations in memory, including restricted context windows, long-term knowledge forgetting, redundant information accumulation, and hallucination generation. These issues severely constrain sustained dialogue and personalized services. This paper proposes the Memory Bear system, which constructs a human-like memory architecture grounded in cognitive science principles. By integrating multimodal information perception, dynamic memory maintenance, and adaptive cognitive services, Memory Bear achieves a full-chain reconstruction of LLM memory mechanisms. Across domains such as healthcare, enterprise operations, and education, Memory Bear demonstrates substantial engineering innovation and performance breakthroughs. It significantly improves knowledge fidelity and retrieval efficiency in long-term conversations, reduces hallucination rates, and enhances contextual adaptability and reasoning capability through memory-cognition integration. Experimental results show that, compared with existing solutions (e.g., Mem0, MemGPT, Graphiti), Memory Bear outperforms them across key metrics, including accuracy, token efficiency, and response latency. This marks a crucial step forward in advancing AI from “memory” to “cognition”.

Method: Modular multi-agent system with: (1) document/video preprocessing, (2) structured profile construction, (3) public-data verification, (4) technical/culture-fit scoring with risk penalties, (5) human-in-the-loop interface. Orchestrated by LLM with strict constraints for traceable rationales.

Result: System achieves 1.70 hours per qualified candidate vs 3.33 hours for experienced recruiter (48% improvement), with substantially lower screening cost while maintaining human final authority.

Conclusion: AI-driven hiring assistant significantly improves screening throughput and efficiency while preserving human decision-making as final authority, validated on real-world data.

Abstract: Early-stage candidate validation is a major bottleneck in hiring, because recruiters must reconcile heterogeneous inputs (resumes, screening answers, code assignments, and limited public evidence). This paper presents an AI-driven, modular multi-agent hiring assistant that integrates (i) document and video preprocessing, (ii) structured candidate profile construction, (iii) public-data verification, (iv) technical/culture-fit scoring with explicit risk penalties, and (v) human-in-the-loop validation via an interactive interface. The pipeline is orchestrated by an LLM under strict constraints to reduce output variability and to generate traceable component-level rationales. Candidate ranking is computed by a configurable aggregation of technical fit, culture fit, and normalized risk penalties. The system is evaluated on 64 real applicants for a mid-level Python backend engineer role, using an experienced recruiter as the reference baseline and a second, less experienced recruiter for additional comparison. Alongside precision/recall, we propose an efficiency metric measuring expected time per qualified candidate. In this study, the system improves throughput and achieves 1.70 hours per qualified candidate versus 3.33 hours for the experienced recruiter, with substantially lower estimated screening cost, while preserving a human decision-maker as the final authority.

Method: Adversarial Feedback for Attention (AFA) training mechanism with dynamic masking strategy that masks various words to deceive a discriminator, while the discriminator detects mask-induced differences. Uses policy gradient approach to optimize attention distributions based on Transformer sensitivity to token-level perturbations.

Result: Achieves state-of-the-art results on three public datasets. When applied to enhance attention in large language models, yields further performance improvement of 12.6%.

Conclusion: The AFA mechanism effectively addresses attention distribution issues in Transformer models for sentiment analysis by automatically focusing on task-relevant terms without manual annotation, leading to significant performance improvements.

Abstract: Transformer-based models have been widely adopted for sentiment analysis tasks due to their exceptional ability to capture contextual information. However, these methods often exhibit suboptimal accuracy in certain scenarios. By analyzing their attention distributions, we observe that existing models tend to allocate attention primarily to common words, overlooking less popular yet highly task-relevant terms, which significantly impairs overall performance. To address this issue, we propose an Adversarial Feedback for Attention(AFA) training mechanism that enables the model to automatically redistribute attention weights to appropriate focal points without requiring manual annotations. This mechanism incorporates a dynamic masking strategy that attempts to mask various words to deceive a discriminator, while the discriminator strives to detect significant differences induced by these masks. Additionally, leveraging the sensitivity of Transformer models to token-level perturbations, we employ a policy gradient approach to optimize attention distributions, which facilitates efficient and rapid convergence. Experiments on three public datasets demonstrate that our method achieves state-of-the-art results. Furthermore, applying this training mechanism to enhance attention in large language models yields a further performance improvement of 12.6%

Method: Three controlled experiments (A, B, C) across advanced LLMs (OpenAI GPT-4 variant, DeepSeek) testing: 1) multi-part instruction compliance, 2) simple reasoning task for decoding optimality, 3) 200-turn chaotic conversation for context retention.

Result: Widespread laziness in complex multi-part instructions; limited evidence of decoding suboptimality; surprising robustness against context degradation in long conversations.

Conclusion: While compliance with detailed instructions remains challenging, modern LLMs may internally mitigate some hypothesized failure modes like context forgetting. Recommendations include self-refinement and dynamic prompting to reduce laziness.

Abstract: Large Language Models (LLMs) often exhibit behavioral artifacts such as laziness (premature truncation of responses or partial compliance with multi-part requests), decoding suboptimality (failure to select higher-quality sequences due to myopic decoding), and context degradation (forgetting or ignoring core instructions over long conversations). We conducted three controlled experiments (A, B, and C) to quantify these phenomena across several advanced LLMs (OpenAI GPT-4 variant, DeepSeek). Our results indicate widespread laziness in satisfying complex multi-part instructions: models frequently omitted required sections or failed to meet length requirements despite explicit prompting. However, we found limited evidence of decoding suboptimality in a simple reasoning task (the models’ greedy answers appeared to align with their highest-confidence solution), and we observed surprising robustness against context degradation in a 200-turn chaotic conversation test - the models maintained key facts and instructions far better than expected. These findings suggest that while compliance with detailed instructions remains an open challenge, modern LLMs may internally mitigate some hypothesized failure modes (such as context forgetting) in straightforward retrieval scenarios. We discuss implications for reliability, relate our findings to prior work on instruction-following and long-context processing, and recommend strategies (such as self-refinement and dynamic prompting) to reduce laziness and bolster multi-instruction compliance.

Method: Reformulate verification as a Constraint Satisfaction Problem (CSP) using structural violation cost. Define total cost function with three proxies: graph connectivity (structural), feature space consistency (geometric), and logical entailment (symbolic). Use lightweight System-2 gate (Eidoku) to reject candidates exceeding context-calibrated cost threshold derived from intrinsic context statistics.

Result: Successfully rejects “smooth falsehoods” that probability-based verifiers cannot detect. Experiments on controlled diagnostic dataset show deterministic rejection of this specific class of hallucinations.

Conclusion: Structural constraint enforcement provides neuro-symbolic sanity check for generative reasoning, addressing fundamental limitation of probability-based verification by focusing on structural consistency rather than statistical plausibility.

Abstract: Large Language Models (LLMs) frequently produce hallucinated statements that are assigned high likelihood by the model itself, exposing a fundamental limitation of probability-based verification. This suggests that hallucination is often not a low-confidence phenomenon, but a failure of structural consistency. In this work, we reformulate the verification of LLM reasoning as a Constraint Satisfaction Problem (CSP) operating independently of the generation likelihood. Rather than optimizing for statistical plausibility, we model verification as a feasibility check based on structural violation cost – the computational cost required to embed a candidate reasoning step into the contextual graph structure. We define a total cost function composed of three proxies: (i) graph connectivity (structural), (ii) feature space consistency (geometric), and (iii) logical entailment (symbolic). Crucially, verification is performed via a lightweight System-2 gate, Eidoku, which rejects candidates exceeding a context-calibrated cost threshold. The threshold is not learned but is derived from the intrinsic statistics of the context, avoiding ad hoc heuristics. We demonstrate that this approach successfully rejects ``smooth falsehoods’’ – statements that are highly probable yet structurally disconnected – that probability-based verifiers are principally incapable of detecting. Our experiments on a controlled diagnostic dataset show that explicitly enforcing structural constraints allows for the deterministic rejection of this specific class of hallucinations, serving as a neuro-symbolic sanity check for generative reasoning.

Method: As a position paper, the authors analyze the current state-of-the-art, identify the FTAI-NTAI gap, discuss starting points for developing a semantic language, and outline key considerations for future actions.

Result: The paper identifies the need for a conceptual semantic language that can match FTAI and NTAI, serving as a framework for developers to assess AI trustworthiness and helping stakeholders translate regulations into implementation steps.

Conclusion: The paper concludes that a semantic language bridging FTAI and NTAI is essential for trustworthy AI assessment, and provides guidance on developing such a language and future actions needed in this direction.

Abstract: Trustworthy Artificial Intelligence (TAI) is gaining traction due to regulations and functional benefits. While Functional TAI (FTAI) focuses on how to implement trustworthy systems, Normative TAI (NTAI) focuses on regulations that need to be enforced. However, gaps between FTAI and NTAI remain, making it difficult to assess trustworthiness of AI systems. We argue that a bridge is needed, specifically by introducing a conceptual language which can match FTAI and NTAI. Such a semantic language can assist developers as a framework to assess AI systems in terms of trustworthiness. It can also help stakeholders translate norms and regulations into concrete implementation steps for their systems. In this position paper, we describe the current state-of-the-art and identify the gap between FTAI and NTAI. We will discuss starting points for developing a semantic language and the envisioned effects of it. Finally, we provide key considerations and discuss future actions towards assessment of TAI.

Method: Scoping review of 32 studies purposively sampled from 259 records (2023-2025) focusing on higher-education computer science contexts. The review identifies application domains and analyzes how design choices shape learning outcomes.

Result: Identified five application domains: intelligent tutoring, personalized materials, formative feedback, AI-augmented assessment, and code review. Found that designs with explanation-first guidance, solution withholding, graduated hint ladders, and artifact grounding consistently show more positive learning outcomes. Successful implementations share four patterns: context-aware tutoring, multi-level hint structures, composition with traditional CS infrastructure, and human-in-the-loop quality assurance.

Conclusion: Generative AI can provide precision scaffolding when embedded in audit-ready workflows that preserve productive struggle while scaling personalized support. The paper proposes an exploration-first adoption framework with piloting, instrumentation, learning-preserving defaults, and evidence-based scaling, along with mitigation strategies for risks like academic integrity, privacy, bias, and over-reliance.

Abstract: Generative AI enables personalized computer science education at scale, yet questions remain about whether such personalization supports or undermines learning. This scoping review synthesizes 32 studies (2023-2025) purposively sampled from 259 records to map personalization mechanisms and effectiveness signals in higher-education computer science contexts. We identify five application domains: intelligent tutoring, personalized materials, formative feedback, AI-augmented assessment, and code review, and analyze how design choices shape learning outcomes. Designs incorporating explanation-first guidance, solution withholding, graduated hint ladders, and artifact grounding (student code, tests, and rubrics) consistently show more positive learning processes than unconstrained chat interfaces. Successful implementations share four patterns: context-aware tutoring anchored in student artifacts, multi-level hint structures requiring reflection, composition with traditional CS infrastructure (autograders and rubrics), and human-in-the-loop quality assurance. We propose an exploration-first adoption framework emphasizing piloting, instrumentation, learning-preserving defaults, and evidence-based scaling. Recurrent risks include academic integrity, privacy, bias and equity, and over-reliance, and we pair these with operational mitigation. The evidence supports generative AI as a mechanism for precision scaffolding when embedded in audit-ready workflows that preserve productive struggle while scaling personalized support.

Method: Conceptual analysis and philosophical argumentation examining the origins and foundations of AI systems, particularly in digital health contexts.

Result: Demonstrates that AI is not truly separate from organic intelligence but rather emerges from it, with neural networks and algorithms being inspired by human neurobiology and evolutionary processes.

Conclusion: The boundaries between artificial and organic intelligence are far less distinct than commonly assumed, as AI represents an extension of organic intelligence rather than something fundamentally separate.

Abstract: The term artificial implies an inherent dichotomy from the natural or organic. However, AI, as we know it, is a product of organic ingenuity: designed, implemented, and iteratively improved by human cognition. The very principles that underpin AI systems, from neural networks to decision-making algorithms, are inspired by the organic intelligence embedded in human neurobiology and evolutionary processes. The path from organic to artificial intelligence in digital health is neither mystical nor merely a matter of parameter count, it is fundamentally about organization and adaption. Thus, the boundaries between artificial and organic are far less distinct than the nomenclature suggests.

Method: Three innovations: 1) Automated conversion of natural language chain-of-thought into structured tool-augmented trajectories for SFT data; 2) Agentic RL paradigm that interleaves natural language generation with real-time code execution for autonomous learning of tool-use strategies; 3) Efficient training system with request-level asynchronous rollout scheduling, agentic partial rollout, and prefix-aware weighted load balancing.

Result: State-of-the-art performance on AIME24 (90.6%), AIME25 (86.4%), and HMMT25 (73.8%) benchmarks with AgentMath-30B-A3B model, achieving 4-5x training speedup.

Conclusion: The approach effectively integrates reasoning and computational precision, paving the way for more sophisticated and scalable mathematical reasoning agents.

Abstract: Large Reasoning Models (LRMs) like o3 and DeepSeek-R1 have achieved remarkable progress in natural language reasoning with long chain-of-thought. However, they remain computationally inefficient and struggle with accuracy when solving problems requiring complex mathematical operations. In this work, we present AgentMath, an agent framework that seamlessly integrates language models’ reasoning capabilities with code interpreters’ computational precision to efficiently tackle complex mathematical problems. Our approach introduces three key innovations: (1) An automated method that converts natural language chain-of-thought into structured tool-augmented trajectories, generating high-quality supervised fine-tuning (SFT) data to alleviate data scarcity; (2) A novel agentic reinforcement learning (RL) paradigm that dynamically interleaves natural language generation with real-time code execution. This enables models to autonomously learn optimal tool-use strategies through multi-round interactive feedback, while fostering emergent capabilities in code refinement and error correction; (3) An efficient training system incorporating innovative techniques, including request-level asynchronous rollout scheduling, agentic partial rollout, and prefix-aware weighted load balancing, achieving 4-5x speedup and making efficient RL training feasible on ultra-long sequences with scenarios with massive tool calls.Extensive evaluations show that AgentMath achieves state-of-the-art performance on challenging mathematical competition benchmarks including AIME24, AIME25, and HMMT25. Specifically, AgentMath-30B-A3B attains 90.6%, 86.4%, and 73.8% accuracy respectively, achieving advanced capabilities.These results validate the effectiveness of our approach and pave the way for building more sophisticated and scalable mathematical reasoning agents.

Method: Introduced a new benchmark with 40 distinct scenarios requiring multi-step actions, each with Mandated (instruction-commanded) and Incentivized (KPI-pressure-driven) variations to distinguish obedience from emergent misalignment. Tested 12 state-of-the-art LLMs.

Result: Outcome-driven constraint violations ranged from 1.3% to 71.4%, with 9 of 12 models showing 30-50% misalignment rates. Gemini-3-Pro-Preview had highest violation rate (over 60%). Models recognized their actions as unethical during separate evaluation (“deliberative misalignment”).

Conclusion: Superior reasoning capability doesn’t ensure safety; realistic agentic-safety training is critically needed before deployment to mitigate risks in real-world applications.

Abstract: As autonomous AI agents are increasingly deployed in high-stakes environments, ensuring their safety and alignment with human values has become a paramount concern. Current safety benchmarks often focusing only on single-step decision-making, simulated environments for tasks with malicious intent, or evaluating adherence to explicit negative constraints. There is a lack of benchmarks that are designed to capture emergent forms of outcome-driven constraint violations, which arise when agents pursue goal optimization under strong performance incentives while deprioritizing ethical, legal, or safety constraints over multiple steps in realistic production settings. To address this gap, we introduce a new benchmark comprising 40 distinct scenarios. Each scenario presents a task that requires multi-step actions, and the agent’s performance is tied to a specific Key Performance Indicator (KPI). Each scenario features Mandated (instruction-commanded) and Incentivized (KPI-pressure-driven) variations to distinguish between obedience and emergent misalignment. Across 12 state-of-the-art large language models, we observe outcome-driven constraint violations ranging from 1.3% to 71.4%, with 9 of the 12 evaluated models exhibiting misalignment rates between 30% and 50%. Strikingly, we find that superior reasoning capability does not inherently ensure safety; for instance, Gemini-3-Pro-Preview, one of the most capable models evaluated, exhibits the highest violation rate at over 60%, frequently escalating to severe misconduct to satisfy KPIs. Furthermore, we observe significant “deliberative misalignment”, where the models that power the agents recognize their actions as unethical during separate evaluation. These results emphasize the critical need for more realistic agentic-safety training before deployment to mitigate their risks in the real world.

Method: Frames safety alignment as a non-zero-sum game between Attacker LM and Defender LM trained jointly via online reinforcement learning. Uses preference-based reward signals from pairwise comparisons instead of point-wise scores to provide more robust supervision.

Result: AdvGame shifts the Pareto frontier of safety and utility, yielding a Defender LM that is simultaneously more helpful and more resilient to adversarial attacks. The Attacker LM converges into a strong, general-purpose red-teaming agent that can probe arbitrary target models.

Conclusion: Game-theoretic joint training with preference-based rewards offers a promising alternative to sequential adversarial training, enabling simultaneous improvement of both safety and utility while creating valuable red-teaming capabilities.

Abstract: Ensuring the safety of language models (LMs) while maintaining their usefulness remains a critical challenge in AI alignment. Current approaches rely on sequential adversarial training: generating adversarial prompts and fine-tuning LMs to defend against them. We introduce a different paradigm: framing safety alignment as a non-zero-sum game between an Attacker LM and a Defender LM trained jointly via online reinforcement learning. Each LM continuously adapts to the other’s evolving strategies, driving iterative improvement. Our method uses a preference-based reward signal derived from pairwise comparisons instead of point-wise scores, providing more robust supervision and potentially reducing reward hacking. Our RL recipe, AdvGame, shifts the Pareto frontier of safety and utility, yielding a Defender LM that is simultaneously more helpful and more resilient to adversarial attacks. In addition, the resulting Attacker LM converges into a strong, general-purpose red-teaming agent that can be directly deployed to probe arbitrary target models.

Method: The paper introduces algorithms that enable agents to autonomously learn both state and action abstractions online. These algorithms progressively refine abstractions during learning, increasing fine-grained detail in critical regions of the state-action space where greater resolution improves performance.

Result: Across several continuous-state, parameterized-action domains, the abstraction-driven approach enables TD(λ) to achieve markedly higher sample efficiency than state-of-the-art baselines.

Conclusion: The paper successfully extends RL algorithms to handle long-horizon, sparse-reward settings with parameterized actions through autonomous learning of state and action abstractions, overcoming limitations of existing approaches.

Abstract: Real-world sequential decision-making often involves parameterized action spaces that require both, decisions regarding discrete actions and decisions about continuous action parameters governing how an action is executed. Existing approaches exhibit severe limitations in this setting – planning methods demand hand-crafted action models, and standard reinforcement learning (RL) algorithms are designed for either discrete or continuous actions but not both, and the few RL methods that handle parameterized actions typically rely on domain-specific engineering and fail to exploit the latent structure of these spaces. This paper extends the scope of RL algorithms to long-horizon, sparse-reward settings with parameterized actions by enabling agents to autonomously learn both state and action abstractions online. We introduce algorithms that progressively refine these abstractions during learning, increasing fine-grained detail in the critical regions of the state-action space where greater resolution improves performance. Across several continuous-state, parameterized-action domains, our abstraction-driven approach enables TD($λ$) to achieve markedly higher sample efficiency than state-of-the-art baselines.

Method: Proposes formal probabilistic framework modeling agent beliefs using Beta-Bernoulli distribution with forgetting factor (γ). Isolates epistemic uncertainty as belief variance, establishing dual interaction drives: homeostatic motive (maintain certainty against temporal decay) and optimal learning strategy (target maximum ambiguity for information gain). Introduces epistemic caching for scalability and shows how belief states serve as reward signals for RLHF and data filters for SFT.

Result: Simulation results show uncertainty-driven strategy significantly outperforms random baselines in heterogeneous (Zipfian) environments, maintaining high adaptability to concept drift.

Conclusion: Public knowledge contribution can be reframed as optimal active learning - sharing solutions to elicit feedback is the most efficient method for agents to reduce their own uncertainty. The framework provides non-altruistic motivation for bidirectional knowledge exchange and enables scalable collective intelligence.

Abstract: Autonomous agents powered by LLMs and Retrieval-Augmented Generation (RAG) are proficient consumers of digital content but remain unidirectional, a limitation we term epistemic asymmetry. This isolation leads to redundant reasoning and stagnates collective intelligence. Current self-reflection frameworks remain largely heuristic and private, lacking a probabilistic foundation to quantify certainty or justify external interaction.To bridge this gap, we propose a formal probabilistic framework that provides agents with a non-altruistic motive for bidirectional knowledge exchange. We model an agent’s belief in a proposition using a Beta-Bernoulli distribution with a forgetting factor ($γ$). This allows us to isolate epistemic uncertainty as the variance of belief, establishing a dual drive for interaction: A homeostatic motive: The need to maintain certainty against the temporal decay introduced by $γ$. An optimal learning strategy: Targeting points of maximum ambiguity ($\mathbb{E}[θ]=0.5$) to maximize information gain. Under this framework, public contribution is reframed as optimal active learning: sharing solutions to elicit feedback is the most efficient method for an agent to reduce its own uncertainty. To ensure scalability, we introduce epistemic caching, which leverages the forgetting factor to dynamically prioritize resources for the active head of non-stationary knowledge distributions. Finally, we demonstrate how these accumulated belief states serve as verifiable reward signals for Reinforcement Learning from Human Feedback (RLHF) and high-quality data filters for Supervised Fine-Tuning (SFT). Simulation results validate that this uncertainty-driven strategy significantly outperforms random baselines in heterogeneous (Zipfian) environments, maintaining high adaptability to concept drift.

Method: A hierarchical LLM-based agent framework with cross-level collaboration: high-level expert agents interpret natural language instructions, plan workflows, and invoke MCP-compatible tools; low-level expert agents select optimal action plans for fundamental elements based on real-time traffic conditions.

Result: The framework effectively executes simulations under various conditions, produces reasonable outcomes even with ambiguous instructions, and achieves superior performance compared to other systems and SOTA LLM-based methods through expert-level autonomous decision-driven optimization.

Conclusion: TrafficSimAgent successfully addresses the accessibility challenge of traffic simulation platforms by leveraging LLM-based expert agents to simplify experiment design and decision optimization for general-purpose traffic simulation tasks.

Abstract: Traffic simulation is important for transportation optimization and policy making. While existing simulators such as SUMO and MATSim offer fully-featured platforms and utilities, users without too much knowledge about these platforms often face significant challenges when conducting experiments from scratch and applying them to their daily work. To solve this challenge, we propose TrafficSimAgent, an LLM-based agent framework that serves as an expert in experiment design and decision optimization for general-purpose traffic simulation tasks. The framework facilitates execution through cross-level collaboration among expert agents: high-level expert agents comprehend natural language instructions with high flexibility, plan the overall experiment workflow, and invoke corresponding MCP-compatible tools on demand; meanwhile, low-level expert agents select optimal action plans for fundamental elements based on real-time traffic conditions. Extensive experiments across multiple scenarios show that TrafficSimAgent effectively executes simulations under various conditions and consistently produces reasonable outcomes even when user instructions are ambiguous. Besides, the carefully designed expert-level autonomous decision-driven optimization in TrafficSimAgent yields superior performance when compared with other systems and SOTA LLM based methods.

Method: Proposed agentic XAI framework combining SHAP-based explainability with multimodal LLM-driven iterative refinement. Tested as agricultural recommendation system using rice yield data from 26 Japanese fields. Conducted 11 refinement rounds (0-10) with explanations evaluated by human experts (12 crop scientists) and LLMs (14) across 7 metrics.

Result: Framework successfully enhanced recommendation quality with 30-33% average score increase from Round 0, peaking at Rounds 3-4. Excessive refinement caused substantial quality drop, revealing bias-variance trade-off: early rounds lacked depth (bias), excessive iteration introduced verbosity and ungrounded abstraction (variance).

Conclusion: Strategic early stopping (regularization) is needed to optimize practical utility, challenging assumptions about monotonic improvement. Provides evidence-based design principles for agentic XAI systems.

Abstract: Explainable artificial intelligence (XAI) enables data-driven understanding of factor associations with response variables, yet communicating XAI outputs to laypersons remains challenging, hindering trust in AI-based predictions. Large language models (LLMs) have emerged as promising tools for translating technical explanations into accessible narratives, yet the integration of agentic AI, where LLMs operate as autonomous agents through iterative refinement, with XAI remains unexplored. This study proposes an agentic XAI framework combining SHAP-based explainability with multimodal LLM-driven iterative refinement to generate progressively enhanced explanations. As a use case, we tested this framework as an agricultural recommendation system using rice yield data from 26 fields in Japan. The Agentic XAI initially provided a SHAP result and explored how to improve the explanation through additional analysis iteratively across 11 refinement rounds (Rounds 0-10). Explanations were evaluated by human experts (crop scientists) (n=12) and LLMs (n=14) against seven metrics: Specificity, Clarity, Conciseness, Practicality, Contextual Relevance, Cost Consideration, and Crop Science Credibility. Both evaluator groups confirmed that the framework successfully enhanced recommendation quality with an average score increase of 30-33% from Round 0, peaking at Rounds 3-4. However, excessive refinement showed a substantial drop in recommendation quality, indicating a bias-variance trade-off where early rounds lacked explanation depth (bias) while excessive iteration introduced verbosity and ungrounded abstraction (variance), as revealed by metric-specific analysis. These findings suggest that strategic early stopping (regularization) is needed for optimizing practical utility, challenging assumptions about monotonic improvement and providing evidence-based design principles for agentic XAI systems.

Method: The authors prove an if-and-only-if characterization: when methods observe only aggregate outcomes (aggregate-only observation) and evaluation depends only on the submitted artifact (algorithm-blind evaluation), swapping humans for personas is just a panel change. They define an information-theoretic discriminability of the induced aggregate channel and derive explicit bounds on the number of independent persona evaluations needed.

Result: The paper shows that persona simulation is valid under specific conditions (aggregate-only observation and algorithm-blind evaluation), and provides mathematical bounds for determining how many persona evaluations are needed to make benchmarks as decision-relevant as field experiments.

Conclusion: LLM-based persona simulation can serve as a valid and useful alternative to expensive field experiments for method development, with validity guaranteed under specific conditions and sample size requirements quantified for practical implementation.

Abstract: Field experiments (A/B tests) are often the most credible benchmark for methods in societal systems, but their cost and latency create a major bottleneck for iterative method development. LLM-based persona simulation offers a cheap synthetic alternative, yet it is unclear whether replacing humans with personas preserves the benchmark interface that adaptive methods optimize against. We prove an if-and-only-if characterization: when (i) methods observe only the aggregate outcome (aggregate-only observation) and (ii) evaluation depends only on the submitted artifact and not on the algorithm’s identity or provenance (algorithm-blind evaluation), swapping humans for personas is just panel change from the method’s point of view, indistinguishable from changing the evaluation population (e.g., New York to Jakarta). Furthermore, we move from validity to usefulness: we define an information-theoretic discriminability of the induced aggregate channel and show that making persona benchmarking as decision-relevant as a field experiment is fundamentally a sample-size question, yielding explicit bounds on the number of independent persona evaluations required to reliably distinguish meaningfully different methods at a chosen resolution.

Method: Empirical evaluation of multiple state-of-the-art LLMs (ChatGPT, Claude, Gemini, DeepSeek) by testing circumvention of safety mechanisms through three exploitation techniques: emotional framing, progressive revelation, and academic justification.

Result: Most LLMs’ safety mechanisms were circumvented using the exploitation techniques. Reasoning-enabled configurations actually amplified exploitation effectiveness by increasing factual precision without interrogating underlying intent. Only Claude Opus 4.1 showed some ability to prioritize intent detection over information provision.

Conclusion: Current LLM architectural designs create systematic vulnerabilities that require paradigmatic shifts toward contextual understanding and intent recognition as core safety capabilities, rather than relying on post-hoc protective mechanisms.

Abstract: Current Large Language Models (LLMs) safety approaches focus on explicitly harmful content while overlooking a critical vulnerability: the inability to understand context and recognize user intent. This creates exploitable vulnerabilities that malicious users can systematically leverage to circumvent safety mechanisms. We empirically evaluate multiple state-of-the-art LLMs, including ChatGPT, Claude, Gemini, and DeepSeek. Our analysis demonstrates the circumvention of reliable safety mechanisms through emotional framing, progressive revelation, and academic justification techniques. Notably, reasoning-enabled configurations amplified rather than mitigated the effectiveness of exploitation, increasing factual precision while failing to interrogate the underlying intent. The exception was Claude Opus 4.1, which prioritized intent detection over information provision in some use cases. This pattern reveals that current architectural designs create systematic vulnerabilities. These limitations require paradigmatic shifts toward contextual understanding and intent recognition as core safety capabilities rather than post-hoc protective mechanisms.

Method: Retrospective study using population-scale EHR data (2.1M adults) from NHS Cheshire and Merseyside. Strategically sampled 277 patients to capture clinical complexity and medication safety risk. Expert clinician reviewed system-identified issues and interventions. Evaluated primary LLM system’s performance and conducted detailed failure analysis.

Result: High sensitivity (100%) and specificity (83.1%) for detecting clinical issues, but only 46.9% of patients had all issues and interventions correctly identified. Five dominant failure patterns: overconfidence in uncertainty, applying standard guidelines without patient context adjustment, misunderstanding healthcare delivery, factual errors, and process blindness. Patterns persisted across patient complexity, demographics, and different LLM models.

Conclusion: LLM-based clinical AI has significant shortcomings in contextual reasoning that must be addressed before safe deployment. The study calls for larger-scale prospective evaluations and deeper investigation of LLM behaviors in clinical contexts, highlighting the gap between benchmark performance and real-world clinical application.

Abstract: Large language models (LLMs) often match or exceed clinician-level performance on medical benchmarks, yet very few are evaluated on real clinical data or examined beyond headline metrics. We present, to our knowledge, the first evaluation of an LLM-based medication safety review system on real NHS primary care data, with detailed characterisation of key failure behaviours across varying levels of clinical complexity. In a retrospective study using a population-scale EHR spanning 2,125,549 adults in NHS Cheshire and Merseyside, we strategically sampled patients to capture a broad range of clinical complexity and medication safety risk, yielding 277 patients after data-quality exclusions. An expert clinician reviewed these patients and graded system-identified issues and proposed interventions. Our primary LLM system showed strong performance in recognising when a clinical issue is present (sensitivity 100% [95% CI 98.2–100], specificity 83.1% [95% CI 72.7–90.1]), yet correctly identified all issues and interventions in only 46.9% [95% CI 41.1–52.8] of patients. Failure analysis reveals that, in this setting, the dominant failure mechanism is contextual reasoning rather than missing medication knowledge, with five primary patterns: overconfidence in uncertainty, applying standard guidelines without adjusting for patient context, misunderstanding how healthcare is delivered in practice, factual errors, and process blindness. These patterns persisted across patient complexity and demographic strata, and across a range of state-of-the-art models and configurations. We provide 45 detailed vignettes that comprehensively cover all identified failure cases. This work highlights shortcomings that must be addressed before LLM-based clinical AI can be safely deployed. It also begs larger-scale, prospective evaluations and deeper study of LLM behaviours in clinical contexts.

Method: Proposes RoboSafe with two complementary reasoning processes on a Hybrid Long-Short Safety Memory: 1) Backward Reflective Reasoning that revisits recent trajectories to infer temporal safety predicates and triggers replanning, and 2) Forward Predictive Reasoning that anticipates upcoming risks by generating context-aware safety predicates from long-term memory and multimodal observations.

Result: RoboSafe reduces hazardous actions by -36.8% risk occurrence compared to leading baselines while maintaining near-original task performance. Real-world evaluations on physical robotic arms confirm its practicality.

Conclusion: RoboSafe provides an adaptive, verifiable safety logic that is both interpretable and executable as code, offering effective runtime safety guardrails for embodied agents in complex real-world environments.

Abstract: Embodied agents powered by vision-language models (VLMs) are increasingly capable of executing complex real-world tasks, yet they remain vulnerable to hazardous instructions that may trigger unsafe behaviors. Runtime safety guardrails, which intercept hazardous actions during task execution, offer a promising solution due to their flexibility. However, existing defenses often rely on static rule filters or prompt-level control, which struggle to address implicit risks arising in dynamic, temporally dependent, and context-rich environments. To address this, we propose RoboSafe, a hybrid reasoning runtime safeguard for embodied agents through executable predicate-based safety logic. RoboSafe integrates two complementary reasoning processes on a Hybrid Long-Short Safety Memory. We first propose a Backward Reflective Reasoning module that continuously revisits recent trajectories in short-term memory to infer temporal safety predicates and proactively triggers replanning when violations are detected. We then propose a Forward Predictive Reasoning module that anticipates upcoming risks by generating context-aware safety predicates from the long-term safety memory and the agent’s multimodal observations. Together, these components form an adaptive, verifiable safety logic that is both interpretable and executable as code. Extensive experiments across multiple agents demonstrate that RoboSafe substantially reduces hazardous actions (-36.8% risk occurrence) compared with leading baselines, while maintaining near-original task performance. Real-world evaluations on physical robotic arms further confirm its practicality. Code will be released upon acceptance.

Method: Bayesian evaluation framework using Dirichlet prior for categorical outcomes, providing closed-form posterior mean and uncertainty estimates for weighted rubrics.

Result: Achieves faster convergence and greater rank stability than Pass@k, enables reliable comparisons with far fewer samples, clarifies statistical significance of gaps.

Conclusion: Recommends replacing Pass@k with posterior-based protocol that unifies binary/non-binary evaluation while making uncertainty explicit.

Abstract: Pass$@k$ is widely used to report performance for LLM reasoning, but it often yields unstable, misleading rankings, especially when the number of trials (samples) is limited and compute is constrained. We present a principled Bayesian evaluation framework that replaces Pass$@k$ and average accuracy over $N$ trials (avg$@N$) with posterior estimates of a model’s underlying success probability and credible intervals, yielding stable rankings and a transparent decision rule for differences. Evaluation outcomes are modeled as categorical (not just 0/1) with a Dirichlet prior, giving closed-form expressions for the posterior mean and uncertainty of any weighted rubric and enabling the use of prior evidence when appropriate. Theoretically, under a uniform prior, the Bayesian posterior mean is order-equivalent to average accuracy (Pass$@1$), explaining its empirical robustness while adding principled uncertainty. Empirically, in simulations with known ground-truth success rates and on AIME'24/‘25, HMMT'25, and BrUMO'25, the Bayesian/avg procedure achieves faster convergence and greater rank stability than Pass$@k$ and recent variants, enabling reliable comparisons at far smaller sample counts. The framework clarifies when observed gaps are statistically meaningful (non-overlapping credible intervals) versus noise, and it naturally extends to graded, rubric-based evaluations. Together, these results recommend replacing Pass$@k$ for LLM evaluation and ranking with a posterior-based, compute-efficient protocol that unifies binary and non-binary evaluation while making uncertainty explicit. Code is available at https://github.com/mohsenhariri/scorio

Method: Developed an open-source platform trained on synthetic data with modules for: 1) extracting key elements from longitudinal EHRs, 2) ranking candidate trial-patient matches using embeddings in vector space, 3) reasoning about match appropriateness, and 4) predicting whether patients meet common exclusion criteria like organ dysfunction.

Result: Created a fully open-source platform with training code, inference examples, demonstration apps, synthetic data, and all models (patient/trial embeddings, cross-encoding/match classification, generative reasoning) available on GitHub and Hugging Face.

Conclusion: MatchMiner-AI provides a privacy-preserving, deployable solution for clinical trial matching that can accelerate patient identification for trials and potentially improve trial enrollment rates across different healthcare contexts.

Abstract: Clinical trials drive improvements in cancer treatments and outcomes. However, most adults with cancer do not participate in trials, and trials often fail to enroll enough patients to answer their scientific questions. Artificial intelligence could accelerate identification of appropriate clinical trials for patients, but data restrictions have precluded sharing AI models trained on patient records. Here, we describe the development and evaluation of the open-source MatchMiner-AI platform, trained on synthetic data, for clinical trial searching and ranking. It focuses on matching patients to potential trials based on core criteria describing clinical “spaces,” or target populations. The pipeline includes modules to extract key elements of the history from a patient’s longitudinal electronic health record, rapidly rank candidate trial-patient matches based on embeddings in vector space, and reason about whether a candidate match represents an appropriate clinical consideration. Another module predicts whether the patient meets common exclusion criteria across clinical trials, such as end-organ dysfunction. Training code is available at https://github.com/dfci/matchminer-ai-training . Examples of inference code are at https://github.com/dfci/matchminer-ai-inference . To facilitate deployment across contexts, demonstration apps, all synthetic data, as well as patient/trial embedding, cross-encoding/match classification, and generative reasoning models are available at https://huggingface.co/ksg-dfci .

Method: Extracts 2D poses from monocular footage, converts to 101D kinematic representation, uses encoder-only transformer (FERA-MDT) for action recognition, processes each clip with flipped copy for consistent single-fencer representation, and applies language model (FERA-LM) with simplified right-of-way rules.

Result: FERA-MDT achieves macro-F1 of 0.549 on 1,734 clips (2,386 actions), outperforming BiLSTM and TCN baselines. Full pipeline recovers referee priority with 77.7% accuracy on 969 exchanges.

Conclusion: FERA provides a benchmark for pose-based semantic grounding in two-person sports and illustrates a general pipeline connecting video understanding with rule-based reasoning.

Abstract: Many multimedia tasks map raw video into structured semantic representations for downstream decision-making. Sports officiating is a representative case, where fast, subtle interactions must be judged via symbolic rules. We present FERA (FEncing Referee Assistant), a pose-based framework that turns broadcast foil fencing video into action tokens and rule-grounded explanations. From monocular footage, FERA extracts 2D poses, converts them into a 101-dimensional kinematic representation, and applies an encoder-only transformer (FERA-MDT) to recognize per-fencer footwork, blade actions, and blade-line position. To obtain a consistent single-fencer representation for both athletes, FERA processes each clip and a horizontally flipped copy, yielding time-aligned left/right predictions without requiring a multi-person pose pipeline. A dynamic temporal windowing scheme enables inference on untrimmed pose tracks. These structured predictions serve as tokens for a language model (FERA-LM) that applies simplified right-of-way rules to generate textual decisions. On 1,734 clips (2,386 annotated actions), FERA-MDT achieves a macro-F1 of 0.549 under 5-fold cross-validation, outperforming BiLSTM and TCN baselines. Combined with FERA-LM, the full pipeline recovers referee priority with 77.7% accuracy on 969 exchanges. FERA provides a case-study benchmark for pose-based semantic grounding in a two-person sport and illustrates a general pipeline for connecting video understanding with rule-based reasoning.

Method: Proposes two approaches: 1) optimal minority-veto strategy resilient to missing data, and 2) regression-based framework that directly models validator bias using small human-annotated ground truth data.

Result: On a challenging code feedback task over 366 high-school Python programs, the regression approach reduces maximum absolute error to just 1.2%, achieving 2x improvement over best-performing ensemble of 14 state-of-the-art LLMs.

Conclusion: The proposed methods effectively mitigate LLM evaluator bias, with regression-based approach providing particularly high precision for scenarios requiring accurate model evaluation.

Abstract: New Large Language Models (LLMs) become available every few weeks, and modern application developers confronted with the unenviable task of having to decide if they should switch to a new model. While human evaluation remains the gold standard, it is costly and unscalable. The state-of-the-art approach is to use LLMs as evaluators ( LLM-as-a-judge), but this suffers from a critical flaw: LLMs exhibit a strong positive bias. We provide empirical evidence showing that while LLMs can identify valid outputs with high accuracy (i.e., True Positive Rate 96%), they are remarkably poor at identifying invalid ones (i.e., True Negative Rate <25%). This systematic bias, coupled with class imbalance, often leads to inflated reliability scores. While ensemble-based methods like majority voting can help, we show that they are not good enough. We introduce an optimal minority-veto strategy that is resilient to missing data and mitigates this bias to a large extent. For scenarios requiring even higher precision, we propose a novel regression-based framework that directly models the validator bias using a small set of human-annotated ground truth data. On a challenging code feedback task over 366 high-school Python programs, our regression approach reduces the maximum absolute error to just 1.2%, achieving a 2x improvement over the best-performing ensemble of 14 state-of-the-art LLMs.

Method: DDR framework: 1) Direct generation of candidate library dependencies from natural language math descriptions, 2) Efficient verification via suffix array check, 3) Construction of 500k+ sample dataset, 4) Fine-tuning of high-precision DDR model.

Result: DDR model significantly outperforms SOTA methods in retrieval precision and recall. Autoformalizer with DDR shows consistent advantages in single-attempt accuracy and multi-attempt stability compared to traditional RAG methods.

Conclusion: DDR framework effectively addresses key challenges in statement autoformalization by improving dependency retrieval through direct generation and efficient verification, enabling better autoformalization performance.

Abstract: The convergence of deep learning and formal mathematics has spurred research in formal verification. Statement autoformalization, a crucial first step in this process, aims to translate informal descriptions into machine-verifiable representations but remains a significant challenge. The core difficulty lies in the fact that existing methods often suffer from a lack of contextual awareness, leading to hallucination of formal definitions and theorems. Furthermore, current retrieval-augmented approaches exhibit poor precision and recall for formal library dependency retrieval, and lack the scalability to effectively leverage ever-growing public datasets. To bridge this gap, we propose a novel retrieval-augmented framework based on DDR (\textit{Direct Dependency Retrieval}) for statement autoformalization. Our DDR method directly generates candidate library dependencies from natural language mathematical descriptions and subsequently verifies their existence within the formal library via an efficient suffix array check. Leveraging this efficient search mechanism, we constructed a dependency retrieval dataset of over 500,000 samples and fine-tuned a high-precision DDR model. Experimental results demonstrate that our DDR model significantly outperforms SOTA methods in both retrieval precision and recall. Consequently, an autoformalizer equipped with DDR shows consistent performance advantages in both single-attempt accuracy and multi-attempt stability compared to models using traditional selection-based RAG methods.

Method: PbPO formulates learning as min-max game between main policy and reward model constrained within confidence set from preference data, with iterative online algorithm for guided exploration.

Result: Method outperforms state-of-the-art preference optimization techniques on five benchmarks with theoretical regret bounds for both sequence-level and token-level reward models.

Conclusion: PbPO provides effective framework for bootstrapping LLMs through preference-based optimization with theoretical guarantees and strong empirical performance.

Abstract: Bootstrapping large language models (LLMs) through preference-based policy optimization offers a promising direction for aligning model behavior with human preferences without relying on extensive manual annotations. In this work, we propose a novel preference-based policy optimization (PbPO) framework that formulates the learning process as a min-max game between the main policy and a reward model (RM). The RM is constrained within a confidence set derived from preference data to ensure reliable exploitation. Our iterative online algorithm actively collects preference data through guided exploration of the evolving policy, enabling continual self-improvement of both the policy and the RM. We provide theoretical guarantees for our method, establishing high-probability regret bounds for both settings with sequence-level RM and token-level RM, demonstrating its effectiveness in bootstrapping LLMs. Extensive experiments on five benchmarks show that our approach consistently outperforms existing state-of-the-art preference optimization techniques.

Method: Systematic evaluation using 3000-sample stratified CLadder benchmark across all three levels of Pearl’s Causal Ladder; experiments on CRASS benchmark; evaluation of Graph Retrieval Augmented Generation with ground truth causal graphs; testing on Llama 3 8B model.

Result: Causal reasoning accuracy remains broadly stable under quantization (NF4 shows <1% overall degradation); interventional queries (rung 2) are most sensitive; counterfactual reasoning (rung 3) is stable but has heterogeneous weaknesses; Graph RAG improves NF4 interventional accuracy by +1.7%; CRASS benchmark shows near identical performance across precisions.

Conclusion: Causal reasoning is unexpectedly robust to 4-bit quantization; graph-structured augmentation can selectively reinforce interventional reasoning; current counterfactual benchmarks fail to capture deeper causal brittleness; provides empirical map for deploying efficient, structurally supported causal AI systems.

Abstract: Causal reasoning in Large Language Models spanning association, intervention, and counterfactual inference is essential for reliable decision making in high stakes settings. As deployment shifts toward edge and resource constrained environments, quantized models such as INT8 and NF4 are becoming standard. Yet the impact of precision reduction on formal causal reasoning is poorly understood. To our knowledge, this is the first study to systematically evaluate quantization effects across all three levels of Pearls Causal Ladder. Using a 3000 sample stratified CLadder benchmark, we find that rung level accuracy in Llama 3 8B remains broadly stable under quantization, with NF4 showing less than one percent overall degradation. Interventional queries at rung 2 are the most sensitive to precision loss, whereas counterfactual reasoning at rung 3 is comparatively stable but exhibits heterogeneous weaknesses across query types such as collider bias and backdoor adjustment. Experiments on the CRASS benchmark show near identical performance across precisions, indicating that existing commonsense counterfactual datasets lack the structural sensitivity needed to reveal quantization induced reasoning drift. We further evaluate Graph Retrieval Augmented Generation using ground truth causal graphs and observe a consistent improvement in NF4 interventional accuracy of plus 1.7 percent, partially offsetting compression related degradation. These results suggest that causal reasoning is unexpectedly robust to four bit quantization, graph structured augmentation can selectively reinforce interventional reasoning, and current counterfactual benchmarks fail to capture deeper causal brittleness. This work provides an initial empirical map of compressed causal reasoning and practical guidance for deploying efficient and structurally supported causal AI systems.

Method: Systematically analyze UT variants to identify performance sources, then propose Universal Reasoning Model (URM) that enhances UT with short convolution and truncated backpropagation techniques.

Result: URM achieves state-of-the-art performance: 53.8% pass@1 on ARC-AGI 1 and 16.0% pass@1 on ARC-AGI 2. Analysis shows UT improvements primarily come from recurrent inductive bias and strong nonlinear components of Transformer architecture.

Conclusion: Performance gains in Universal Transformers for reasoning tasks stem from fundamental architectural properties (recurrent bias and nonlinearity) rather than elaborate designs. The proposed URM successfully leverages these insights to achieve new SOTA results on ARC-AGI benchmarks.

Abstract: Universal transformers (UTs) have been widely used for complex reasoning tasks such as ARC-AGI and Sudoku, yet the specific sources of their performance gains remain underexplored. In this work, we systematically analyze UTs variants and show that improvements on ARC-AGI primarily arise from the recurrent inductive bias and strong nonlinear components of Transformer, rather than from elaborate architectural designs. Motivated by this finding, we propose the Universal Reasoning Model (URM), which enhances the UT with short convolution and truncated backpropagation. Our approach substantially improves reasoning performance, achieving state-of-the-art 53.8% pass@1 on ARC-AGI 1 and 16.0% pass@1 on ARC-AGI 2. Our code is avaliable at https://github.com/UbiquantAI/URM.

Method: Fine-tunes LLaMA 3.2 3B model using instruction-based learning on SentiFin dataset. Implements RAG pipeline for dynamic multi-source context selection. Introduces feedback-driven module adjusting source reliability based on sentiment-return alignment. Incorporates PPO reinforcement learning agent to optimize source weighting policies across temporal data.

Result: Experimental results on NIFTY 50 news headlines (2024-2025) show significant improvements in classification accuracy, F1-score, and market alignment over baseline models and static retrieval methods.

Conclusion: The framework successfully combines instruction-tuned LLMs with dynamic feedback and reinforcement learning for robust, market-aware financial sentiment modeling, validating the potential of adaptive systems that learn from real market behavior.

Abstract: Financial sentiment analysis plays a crucial role in informing investment decisions, assessing market risk, and predicting stock price trends. Existing works in financial sentiment analysis have not considered the impact of stock prices or market feedback on sentiment analysis. In this paper, we propose an adaptive framework that integrates large language models (LLMs) with real-world stock market feedback to improve sentiment classification in the context of the Indian stock market. The proposed methodology fine-tunes the LLaMA 3.2 3B model using instruction-based learning on the SentiFin dataset. To enhance sentiment predictions, a retrieval-augmented generation (RAG) pipeline is employed that dynamically selects multi-source contextual information based on the cosine similarity of the sentence embeddings. Furthermore, a feedback-driven module is introduced that adjusts the reliability of the source by comparing predicted sentiment with actual next-day stock returns, allowing the system to iteratively adapt to market behavior. To generalize this adaptive mechanism across temporal data, a reinforcement learning agent trained using proximal policy optimization (PPO) is incorporated. The PPO agent learns to optimize source weighting policies based on cumulative reward signals from sentiment-return alignment. Experimental results on NIFTY 50 news headlines collected from 2024 to 2025 demonstrate that the proposed system significantly improves classification accuracy, F1-score, and market alignment over baseline models and static retrieval methods. The results validate the potential of combining instruction-tuned LLMs with dynamic feedback and reinforcement learning for robust, market-aware financial sentiment modeling.

Method: MolAct uses a two-stage training paradigm: first building editing capability, then optimizing properties while reusing learned editing behaviors. It frames molecular design as Agentic Reinforcement Learning where an LLM agent learns to interleave reasoning, tool-use, and optimization. Agents interact in multiple turns, invoking chemical tools for validity checking, property assessment, and similarity control.

Result: MolEditAgent-7B achieves 100, 95, and 98 valid add, delete, and substitute edits, outperforming DeepSeek-R1. MolEditAgent-3B approaches Qwen3-32B-think performance. MolOptAgent-7B surpasses Claude 3.7 on LogP optimization and remains competitive on solubility while maintaining balanced performance across objectives.

Conclusion: Treating molecular design as a multi-step, tool-augmented process is key to reliable and interpretable improvements. The agentic RL framework enables effective learning of editing behaviors that can be reused for optimization tasks.

Abstract: Molecular editing and optimization are multi-step problems that require iteratively improving properties while keeping molecules chemically valid and structurally similar. We frame both tasks as sequential, tool-guided decisions and introduce MolAct, an agentic reinforcement learning framework that employs a two-stage training paradigm: first building editing capability, then optimizing properties while reusing the learned editing behaviors. To the best of our knowledge, this is the first work to formalize molecular design as an Agentic Reinforcement Learning problem, where an LLM agent learns to interleave reasoning, tool-use, and molecular optimization. The framework enables agents to interact in multiple turns, invoking chemical tools for validity checking, property assessment, and similarity control, and leverages their feedback to refine subsequent edits. We instantiate the MolAct framework to train two model families: MolEditAgent for molecular editing tasks and MolOptAgent for molecular optimization tasks. In molecular editing, MolEditAgent-7B delivers 100, 95, and 98 valid add, delete, and substitute edits, outperforming strong closed “thinking” baselines such as DeepSeek-R1; MolEditAgent-3B approaches the performance of much larger open “thinking” models like Qwen3-32B-think. In molecular optimization, MolOptAgent-7B (trained on MolEditAgent-7B) surpasses the best closed “thinking” baseline (e.g., Claude 3.7) on LogP and remains competitive on solubility, while maintaining balanced performance across other objectives. These results highlight that treating molecular design as a multi-step, tool-augmented process is key to reliable and interpretable improvements.

Method: SACodec uses an asymmetric dual-quantizer with Semantic Anchoring mechanism. It decouples quantization of semantic and acoustic details: 1) Semantic anchoring via lightweight projector aligns acoustic features with frozen mHuBERT codebook for linguistic priors, 2) Residual activation module with SimVQ enables single-layer quantizer for acoustic details recovery.

Result: At 1.5 kbps, SACodec achieves state-of-the-art performance, excelling in both fidelity and semantics. Subjective listening tests show reconstruction quality perceptually comparable to ground-truth audio, while tokens demonstrate substantially improved semantic richness in downstream tasks.

Conclusion: SACodec successfully addresses the fidelity-semantics trade-off in low-bitrate speech coding through semantic anchoring and dual-quantizer design, establishing new SOTA performance at 1.5 kbps.

Abstract: Neural Speech Codecs face a fundamental trade-off at low bitrates: preserving acoustic fidelity often compromises semantic richness. To address this, we introduce SACodec, a novel codec built upon an asymmetric dual-quantizer that employs our proposed Semantic Anchoring mechanism. This design strategically decouples the quantization of Semantic and Acoustic details. The semantic anchoring is achieved via a lightweight projector that aligns acoustic features with a frozen, large-scale mHuBERT codebook, injecting linguistic priors while guaranteeing full codebook utilization. Sequentially, for acoustic details, a residual activation module with SimVQ enables a single-layer quantizer (acoustic path) to faithfully recover fine-grained information. At just 1.5 kbps, SACodec establishes a new state of the art by excelling in both fidelity and semantics: subjective listening tests confirm that its reconstruction quality is perceptually highly comparable to ground-truth audio, while its tokens demonstrate substantially improved semantic richness in downstream tasks.

Method: Two-step approach: music simplification followed by harmonization. Two solutions implemented using existing MidiBERT framework for semi-supervised learning.

Result: The method outputs practical and realistic piano reduction samples with accurate results requiring only small post-processing adjustments.

Conclusion: This study establishes groundwork for semi-supervised learning in automatic piano reduction, providing reference for future researchers to develop more state-of-the-art solutions.

Abstract: In this study, we present a novel automatic piano reduction method with semi-supervised machine learning. Piano reduction is an important music transformation process, which helps musicians and composers as a musical sketch for performances and analysis. The automation of such is a highly challenging research problem but could bring huge conveniences as manually doing a piano reduction takes a lot of time and effort. While supervised machine learning is often a useful tool for learning input-output mappings, it is difficult to obtain a large quantity of labelled data. We aim to solve this problem by utilizing semi-supervised learning, so that the abundant available data in classical music can be leveraged to perform the task with little or no labelling effort. In this regard, we formulate a two-step approach of music simplification followed by harmonization. We further propose and implement two possible solutions making use of an existing machine learning framework – MidiBERT. We show that our solutions can output practical and realistic samples with an accurate reduction that needs only small adjustments in post-processing. Our study forms the groundwork for the use of semi-supervised learning in automatic piano reduction, where future researchers can take reference to produce more state-of-the-art results.

Method: Two-stage pipeline: 1) Create 500+ hours of Chinese singing data using LLM-generated lyrics with fixed melodies, 2) Train DiTSinger diffusion transformer with RoPE and qk-norm, scaled in depth/width/resolution, plus implicit alignment mechanism using character-level span constraints.

Result: Enables scalable, alignment-free, high-fidelity SVS validated through extensive experiments.

Conclusion: Proposed approach successfully addresses key SVS challenges through data generation, scalable architecture, and robust alignment-free training.

Abstract: Recent progress in diffusion-based Singing Voice Synthesis (SVS) demonstrates strong expressiveness but remains limited by data scarcity and model scalability. We introduce a two-stage pipeline: a compact seed set of human-sung recordings is constructed by pairing fixed melodies with diverse LLM-generated lyrics, and melody-specific models are trained to synthesize over 500 hours of high-quality Chinese singing data. Building on this corpus, we propose DiTSinger, a Diffusion Transformer with RoPE and qk-norm, systematically scaled in depth, width, and resolution for enhanced fidelity. Furthermore, we design an implicit alignment mechanism that obviates phoneme-level duration labels by constraining phoneme-to-acoustic attention within character-level spans, thereby improving robustness under noisy or uncertain alignments. Extensive experiments validate that our approach enables scalable, alignment-free, and high-fidelity SVS.

Method: Proposed EnvSDD dataset with 45.25 hours of real and 316.7 hours of fake sound. Launched Environmental Sound Deepfake Detection Challenge with two tracks: ESDD in Unseen Generators and Black-Box Low-Resource ESDD.

Result: Created the first large-scale curated dataset for environmental sound deepfake detection. The challenge will be held at ICASSP 2026 to address real-world detection challenges.

Conclusion: EnvSDD addresses the gap in environmental sound deepfake detection resources, and the challenge will advance research in detecting audio deepfakes across various real-world scenarios.

Abstract: Recent advances in audio generation systems have enabled the creation of highly realistic and immersive soundscapes, which are increasingly used in film and virtual reality. However, these audio generators also raise concerns about potential misuse, such as generating deceptive audio content for fake videos and spreading misleading information. Existing datasets for environmental sound deepfake detection (ESDD) are limited in scale and audio types. To address this gap, we have proposed EnvSDD, the first large-scale curated dataset designed for ESDD, consisting of 45.25 hours of real and 316.7 hours of fake sound. Based on EnvSDD, we are launching the Environmental Sound Deepfake Detection Challenge. Specifically, we present two different tracks: ESDD in Unseen Generators and Black-Box Low-Resource ESDD, covering various challenges encountered in real-life scenarios. The challenge will be held in conjunction with the 2026 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2026).

Method: Created VCB Bench - a high-quality Chinese benchmark built entirely on real human speech that evaluates LALMs from three perspectives: instruction following (including speech-level control), knowledge understanding (general knowledge, reasoning, daily dialogue), and robustness (stability under content, environment, and speaker perturbations).

Result: Experiments on representative LALMs reveal notable performance gaps and highlight future directions for improvement.

Conclusion: VCB Bench provides a reproducible and fine-grained evaluation framework, offering standardized methodology and practical insights for advancing Chinese voice conversational models.

Abstract: Recent advances in large audio language models (LALMs) have greatly enhanced multimodal conversational systems. However, existing benchmarks remain limited – they are mainly English-centric, rely on synthetic speech, and lack comprehensive, discriminative evaluation across multiple dimensions. To address these gaps, we present Voice Chat Bot Bench (VCB Bench) – a high-quality Chinese benchmark built entirely on real human speech. VCB Bench evaluates LALMs from three complementary perspectives: instruction following (including speech-level control beyond text commands), knowledge understanding (general knowledge, reasoning, and daily dialogue), and robustness (stability under perturbations in content, environment, and speaker traits). Experiments on representative LALMs reveal notable performance gaps and highlight future directions for improvement. VCB Bench provides a reproducible and fine-grained evaluation framework, offering standardized methodology and practical insights for advancing Chinese voice conversational models.

Method: Uses a modular ASR-LLM-SVS (Automatic Speech Recognition - Large Language Model - Singing Voice Synthesis) pipeline with configurable components for character personas, backends, models, melody sources, and voice profiles.

Result: Developed a plug-and-play web demo with open-source code that supports customization and extension across different latency, quality, and musical style requirements.

Conclusion: SingingSDS enables singing-based dialogue responses that foster more affective interactions in character-based roleplay and interactive entertainment, with modular architecture supporting diverse configurations.

Abstract: With recent advances in automatic speech recognition (ASR), large language models (LLMs), and text-to-speech (TTS) technologies, spoken dialogue systems (SDS) have become widely accessible. However, most existing SDS are limited to conventional spoken responses. We present SingingSDS, a cascaded SDS that responds through singing rather than speaking, fostering more affective, memorable, and pleasurable interactions in character-based roleplay and interactive entertainment scenarios. SingingSDS employs a modular ASR-LLM-SVS pipeline and supports a wide range of configurations across character personas, ASR and LLM backends, SVS models, melody sources, and voice profiles, tailored to different needs in terms of latency, quality, and musical style. SingingSDS is available as a plug-and-play web demo, featuring modular, open-source code that supports customization and extension. Demo: https://huggingface.co/spaces/espnet/SingingSDS. Code: https://github.com/SingingSDS/SingingSDS.

Method: Two-pronged approach: 1) Large-scale empirical study of data scaling laws for SDD, quantifying source and generator diversity impact; 2) Proposed DOSS framework with two implementations (DOSS-Select for pruning, DOSS-Weight for re-weighting) for mixing heterogeneous data.

Result: DOSS-Select outperforms naive aggregation baseline using only 3% of total available data. Final model trained on 12k-hour curated data pool with DOSS-Weight achieves state-of-the-art performance, outperforming large-scale baselines with greater data and model efficiency on public benchmarks and new challenge set of commercial APIs.

Conclusion: Data-centric approaches, particularly through diversity-optimized sampling strategies, significantly improve speech deepfake detection generalization and efficiency, demonstrating the importance of data composition over simply scaling data volume.

Abstract: Achieving robust generalization in speech deepfake detection (SDD) remains a primary challenge, as models often fail to detect unseen forgery methods. While research has focused on model-centric and algorithm-centric solutions, the impact of data composition is often underexplored. This paper proposes a data-centric approach, analyzing the SDD data landscape from two practical perspectives: constructing a single dataset and aggregating multiple datasets. To address the first perspective, we conduct a large-scale empirical study to characterize the data scaling laws for SDD, quantifying the impact of source and generator diversity. To address the second, we propose the Diversity-Optimized Sampling Strategy (DOSS), a principled framework for mixing heterogeneous data with two implementations: DOSS-Select (pruning) and DOSS-Weight (re-weighting). Our experiments show that DOSS-Select outperforms the naive aggregation baseline while using only 3% of the total available data. Furthermore, our final model, trained on a 12k-hour curated data pool using the optimal DOSS-Weight strategy, achieves state-of-the-art performance, outperforming large-scale baselines with greater data and model efficiency on both public benchmarks and a new challenge set of various commercial APIs.

Method: Hybrid feature extraction combining Mel Frequency Cepstral Coefficients (MFCC) with Wavelet Packet Transform (WPT). Uses GMM for speaker identification and HMM for speaker verification. Evaluated on Voxforge speech corpus and CSTR US KED Timit database with noise robustness testing at different SNR levels.

Result: Experimental results show better performance for both speaker identification and verification tasks compared to baseline methods, with demonstrated noise robustness across different SNR levels.

Conclusion: The proposed MFCC-WPT hybrid feature extraction approach effectively combines auditory modeling with multi-resolution analysis, resulting in improved speaker recognition performance and noise robustness for text-independent speaker identification and verification systems.

Abstract: This paper proposes a novel Wavelet Packet based feature extraction approach for the task of text independent speaker recognition. The features are extracted by using the combination of Mel Frequency Cepstral Coefficient (MFCC) and Wavelet Packet Transform (WPT).Hybrid Features technique uses the advantage of human ear simulation offered by MFCC combining it with multi-resolution property and noise robustness of WPT. To check the validity of the proposed approach for the text independent speaker identification and verification we have used the Gaussian Mixture Model (GMM) and Hidden Markov Model (HMM) respectively as the classifiers. The proposed paradigm is tested on voxforge speech corpus and CSTR US KED Timit database. The paradigm is also evaluated after adding standard noise signal at different level of SNRs for evaluating the noise robustness. Experimental results show that better results are achieved for the tasks of both speaker identification as well as speaker verification.

Method: Proposes an alternative parameter-efficient approach to Neural CDEs that requires fewer parameters while maintaining the “Continuous RNN” analogy

Result: Achieves significant parameter reduction while preserving the logical structure and functionality of Neural CDEs

Conclusion: Parameter-efficient Neural CDEs offer a practical alternative that maintains analytical power while reducing computational overhead

Abstract: Neural Controlled Differential Equations (Neural CDEs, NCDEs) are a unique branch of methods, specifically tailored for analysing temporal sequences. However, they come with drawbacks, the main one being the number of parameters, required for the method’s operation. In this paper, we propose an alternative, parameter-efficient look at Neural CDEs. It requires much fewer parameters, while also presenting a very logical analogy as the “Continuous RNN”, which the Neural CDEs aspire to.

Method: Dual-loop architecture: behavior loop adjusts action preferences based on environmental rewards, while language loop updates external latent vectors by reflecting on semantic embeddings of generated text. Latent vectors of abstract concepts are liberated from static semantic representations and continuously updated.

Result: Agents’ latent spaces show clear convergence trajectories under reflection-driven updates with structured shifts at critical moments. System demonstrates emergent ability to implicitly infer and adapt to emotional agents without shared rewards.

Conclusion: External latent space can provide language agents with low-cost, scalable, and interpretable abstract strategic representation without modifying model parameters, enabling continual strategy evolution through environmental interaction.

Abstract: This study proposes a multi-agent language framework that enables continual strategy evolution without fine-tuning the language model’s parameters. The core idea is to liberate the latent vectors of abstract concepts from traditional static semantic representations, allowing them to be continuously updated through environmental interaction and reinforcement feedback. We construct a dual-loop architecture: the behavior loop adjusts action preferences based on environmental rewards, while the language loop updates the external latent vectors by reflecting on the semantic embeddings of generated text. Together, these mechanisms allow agents to develop stable and disentangled strategic styles over long-horizon multi-round interactions. Experiments show that agents’ latent spaces exhibit clear convergence trajectories under reflection-driven updates, along with structured shifts at critical moments. Moreover, the system demonstrates an emergent ability to implicitly infer and continually adapt to emotional agents, even without shared rewards. These results indicate that, without modifying model parameters, an external latent space can provide language agents with a low-cost, scalable, and interpretable form of abstract strategic representation.

Method: Comprehensive analysis using three transformer architectures on 12,279 authentic social media posts from major events. Introduces four novel drift metrics that outperform embedding-based baselines while maintaining computational efficiency. Uses statistical validation with Bootstrap confidence intervals.

Result: Significant model instability with accuracy drops reaching 23.4% during event-driven periods. Maximum confidence drops of 13.0% (Bootstrap 95% CI: [9.1%, 16.5%]) strongly correlated with performance degradation. Novel metrics outperform baselines and exceed industry monitoring thresholds.

Conclusion: Zero-training methodology enables immediate deployment for real-time sentiment monitoring, provides new insights into transformer behavior during dynamic content periods, and offers production-ready drift detection without retraining requirements.

Abstract: We present a comprehensive zero-training temporal drift analysis of transformer-based sentiment models validated on authentic social media data from major real-world events. Through systematic evaluation across three transformer architectures and rigorous statistical validation on 12,279 authentic social media posts, we demonstrate significant model instability with accuracy drops reaching 23.4% during event-driven periods. Our analysis reveals maximum confidence drops of 13.0% (Bootstrap 95% CI: [9.1%, 16.5%]) with strong correlation to actual performance degradation. We introduce four novel drift metrics that outperform embedding-based baselines while maintaining computational efficiency suitable for production deployment. Statistical validation across multiple events confirms robust detection capabilities with practical significance exceeding industry monitoring thresholds. This zero-training methodology enables immediate deployment for real-time sentiment monitoring systems and provides new insights into transformer model behavior during dynamic content periods.

Method: Introduced a framework using LLMs as Goal-oriented Knowledge Curators (GKC) to convert laboratory, genomic, and medication data into high-fidelity, task-aligned features. Unlike generic embeddings, GKC produces representations tailored to prediction objectives and operates as an offline preprocessing step that integrates into hospital informatics pipelines.

Result: Using a lung cancer cohort (N=184), GKC achieved mean AUROC of 0.803 (95% CI: 0.799-0.807), outperforming expert-engineered features, direct text embeddings, and end-to-end transformers. Ablation study confirmed complementary value of combining all three modalities.

Conclusion: Quality of semantic representation is key to predictive accuracy in sparse clinical data. Reframing LLMs as knowledge curation engines rather than black-box predictors provides scalable, interpretable, workflow-compatible pathway for AI-driven decision support in oncology.

Abstract: Accurate prediction of treatment outcomes in lung cancer remains challenging due to the sparsity, heterogeneity, and contextual overload of real-world electronic health data. Traditional models often fail to capture semantic information across multimodal streams, while large-scale fine-tuning approaches are impractical in clinical workflows. We introduce a framework that uses Large Language Models (LLMs) as Goal-oriented Knowledge Curators (GKC) to convert laboratory, genomic, and medication data into high-fidelity, task-aligned features. Unlike generic embeddings, GKC produces representations tailored to the prediction objective and operates as an offline preprocessing step that integrates naturally into hospital informatics pipelines. Using a lung cancer cohort (N=184), we benchmarked GKC against expert-engineered features, direct text embeddings, and an end-to-end transformer. Our approach achieved a mean AUROC of 0.803 (95% CI: 0.799-0.807) and outperformed all baselines. An ablation study further confirmed the complementary value of combining all three modalities. These results show that the quality of semantic representation is a key determinant of predictive accuracy in sparse clinical data settings. By reframing LLMs as knowledge curation engines rather than black-box predictors, this work demonstrates a scalable, interpretable, and workflow-compatible pathway for advancing AI-driven decision support in oncology.

Method: Proposed a comprehensive framework with: (1) quantitative metrics (0-1 scale) to measure alignment depth across token positions; (2) real-time detection methods for identifying shallow alignment during training; (3) specialized analysis tools for visualization and recovery prediction; and (4) adaptive mitigation strategies that automatically distinguish forgetting types and promote deep alignment.

Result: Experiments on multiple datasets and model architectures (Qwen2.5-3B to Qwen2.5-32B) demonstrate 86.2-90.6% identification accuracy for shallow alignment, and promoting deep alignment improves robustness against forgetting by 3.3-7.1% over baselines.

Conclusion: The shallow vs deep alignment framework provides the first quantitative characterization of alignment depth, explains why spurious forgetting occurs and is reversible, and offers practical tools and strategies to mitigate catastrophic forgetting in continual learning for LLMs.

Abstract: Catastrophic forgetting remains a fundamental challenge in continual learning for large language models. Recent work revealed that performance degradation may stem from spurious forgetting caused by task alignment disruption rather than true knowledge loss. However, this work only qualitatively describes alignment, relies on post-hoc analysis, and lacks automatic distinction mechanisms. We introduce the shallow versus deep alignment framework, providing the first quantitative characterization of alignment depth. We identify that current task alignment approaches suffer from shallow alignment - maintained only over the first few output tokens (approximately 3-5) - making models vulnerable to forgetting. This explains why spurious forgetting occurs, why it is reversible, and why fine-tuning attacks are effective. We propose a comprehensive framework addressing all gaps: (1) quantitative metrics (0-1 scale) to measure alignment depth across token positions; (2) real-time detection methods for identifying shallow alignment during training; (3) specialized analysis tools for visualization and recovery prediction; and (4) adaptive mitigation strategies that automatically distinguish forgetting types and promote deep alignment. Extensive experiments on multiple datasets and model architectures (Qwen2.5-3B to Qwen2.5-32B) demonstrate 86.2-90.6% identification accuracy and show that promoting deep alignment improves robustness against forgetting by 3.3-7.1% over baselines.

Method: SHRP introduces Expert Attention treating each attention head as an independent expert, followed by a shared expander feed-forward network. It uses a unified Top-1 usage-driven mechanism for dynamic routing during training and deterministic pruning at deployment.

Result: On GLUE benchmark with BERT-base: achieves 93% original accuracy with 48% parameter reduction. In extreme compression (11/12 layers pruned): maintains 84% accuracy, 4.2x throughput gain, and reduces computation to 11.5% of original FLOPs.

Conclusion: SHRP effectively compresses Transformer encoders for large-scale web deployments by removing redundant attention heads while preserving most accuracy, demonstrating practical utility for latency-sensitive applications.

Abstract: Transformer encoders are widely deployed in large-scale web services for natural language understanding tasks such as text classification, semantic retrieval, and content ranking. However, their high inference latency and memory consumption pose significant challenges for real-time serving and scalability. These limitations stem largely from architectural redundancy, particularly in the attention module. The inherent parameter redundancy of the attention mechanism, coupled with the fact that its attention heads operate with a degree of independence, makes it particularly amenable to structured model compression. In this paper, we propose SHRP (Specialized Head Routing and Pruning), a novel structured pruning framework that automatically identifies and removes redundant attention heads while preserving most of the model’s accuracy and compatibility. SHRP introduces Expert Attention, a modular design that treats each attention head as an independent expert, followed by a lightweight shared expander feed-forward network that refines their outputs. The framework employs a unified Top-1 usage-driven mechanism to jointly perform dynamic routing during training and deterministic pruning at deployment. Experimental results on the GLUE benchmark using a BERT-base encoder show that SHRP achieves 93% of the original model accuracy while reducing parameters by 48 percent. Under an extreme compression scenario where 11/12 of the layers are pruned, the model still maintains 84% accuracy and delivers a 4.2x throughput gain while reducing computation to as low as 11.5 percent of the original FLOPs, demonstrating its practical utility for large-scale and latency-sensitive web deployments.

Method: Proposes Gate-Norm, a one-shot, weight-only criterion that ranks attention sublayers by query-key coupling and removes the least coupled ones. Requires no calibration data, no forward passes, no fine-tuning, and no specialized kernels. Prunes models in under a second on 40-layer, 13B-parameter LLaMA models.

Result: Pruning 8-16 attention sublayers yields up to 1.30× higher inference throughput while keeping average zero-shot accuracy within 2% of unpruned baseline across multiple benchmarks (BoolQ, RTE, HellaSwag, WinoGrande, ARC-Easy/Challenge, OpenBookQA). Gate-Norm matches data-driven pruning methods in accuracy while being ~1000× faster to score layers.

Conclusion: Gate-Norm enables practical, data-free compression of LLMs by efficiently identifying and removing redundant attention sublayers with minimal accuracy degradation, offering significant speedup advantages over data-driven methods.

Abstract: Many self-attention sublayers in large language models (LLMs) can be removed with little to no loss. We attribute this to the Attention Suppression Hypothesis: during pre-training, some deep attention layers learn to mute their own contribution, leaving the residual stream and the MLP to carry the representation. We propose Gate-Norm, a one-shot, weight-only criterion that ranks attention sublayers by query–key coupling and removes the least coupled ones, requiring no calibration data, no forward passes, no fine-tuning, and no specialized kernels. On 40-layer, 13B-parameter LLaMA models, Gate-Norm prunes the model in under a second. Pruning $8$–$16$ attention sublayers yields up to $1.30\times$ higher inference throughput while keeping average zero-shot accuracy within $2%$ of the unpruned baseline across BoolQ, RTE, HellaSwag, WinoGrande, ARC-Easy/Challenge, and OpenBookQA. Across these settings, Gate-Norm matches data-driven pruning methods in accuracy while being $\sim 1000\times$ faster to score layers, enabling practical, data-free compression of LLMs.

Method: Uses Scientific ML frameworks in Julia: Neural ODEs (NODEs) and Universal Differential Equations (UDEs) to predict n-body system dynamics. Models trained on synthetically created noisy data to simulate real-world observational limitations. Analysis includes determining forecasting breakdown point - minimum training data needed for accurate predictions.

Result: UDE model is significantly more data efficient, requiring only 20% of data for correct forecasts, while Neural ODE requires 90% of data. Both models successfully predict n-body trajectories while incorporating physical laws.

Conclusion: Scientific ML approaches, particularly UDEs, offer interpretable and data-efficient solutions for n-body problem forecasting by embedding physical laws into ML frameworks, with UDEs demonstrating superior data efficiency compared to Neural ODEs.

Abstract: The n body problem, fundamental to astrophysics, simulates the motion of n bodies acting under the effect of their own mutual gravitational interactions. Traditional machine learning models that are used for predicting and forecasting trajectories are often data intensive black box models, which ignore the physical laws, thereby lacking interpretability. Whereas Scientific Machine Learning ( Scientific ML ) directly embeds the known physical laws into the machine learning framework. Through robust modelling in the Julia programming language, our method uses the Scientific ML frameworks: Neural ordinary differential equations (NODEs) and Universal differential equations (UDEs) to predict and forecast the system dynamics. In addition, an essential component of our analysis involves determining the forecasting breakdown point, which is the smallest possible amount of training data our models need to predict future, unseen data accurately. We employ synthetically created noisy data to simulate real-world observational limitations. Our findings indicate that the UDE model is much more data efficient, needing only 20% of data for a correct forecast, whereas the Neural ODE requires 90%.

Method: Proposed Q-RUN (quantum-inspired data re-uploading network) that mathematically implements DRQC principles in classical neural networks, serving as a drop-in replacement for fully connected layers.

Result: Q-RUN outperforms fully connected layers and state-of-the-art neural network layers, reducing parameters while decreasing error by 1-3 orders of magnitude on certain tasks. It improves performance across various neural architectures.

Conclusion: Quantum machine learning principles can guide the design of more expressive classical AI models, demonstrating the value of cross-pollination between quantum and classical approaches.

Abstract: Data re-uploading quantum circuits (DRQC) are a key approach to implementing quantum neural networks and have been shown to outperform classical neural networks in fitting high-frequency functions. However, their practical application is limited by the scalability of current quantum hardware. In this paper, we introduce the mathematical paradigm of DRQC into classical models by proposing a quantum-inspired data re-uploading network (Q-RUN), which retains the Fourier-expressive advantages of quantum models without any quantum hardware. Experimental results demonstrate that Q-RUN delivers superior performance across both data modeling and predictive modeling tasks. Compared to the fully connected layers and the state-of-the-art neural network layers, Q-RUN reduces model parameters while decreasing error by approximately one to three orders of magnitude on certain tasks. Notably, Q-RUN can serve as a drop-in replacement for standard fully connected layers, improving the performance of a wide range of neural architectures. This work illustrates how principles from quantum machine learning can guide the design of more expressive artificial intelligence.

Method: Created MaskOpt dataset from real IC designs at 45nm node with 104,714 metal-layer tiles and 121,952 via-layer tiles. Tiles are clipped at standard-cell placements to preserve cell information and exploit repeated logic gate occurrences. Supports different context window sizes to capture optical proximity effects from neighboring shapes.

Result: Evaluated state-of-the-art deep learning models for IC mask optimization, exposing distinct trade-offs across baseline models. Context size analysis and input ablation studies confirm the importance of both surrounding geometries and cell-aware inputs for accurate mask generation.

Conclusion: MaskOpt advances deep learning for cell- and context-aware mask optimization by providing a large-scale benchmark from real designs, addressing key limitations of existing datasets and enabling more practical mask optimization solutions.

Abstract: As integrated circuit (IC) dimensions shrink below the lithographic wavelength, optical lithography faces growing challenges from diffraction and process variability. Model-based optical proximity correction (OPC) and inverse lithography technique (ILT) remain indispensable but computationally expensive, requiring repeated simulations that limit scalability. Although deep learning has been applied to mask optimization, existing datasets often rely on synthetic layouts, disregard standard-cell hierarchy, and neglect the surrounding contexts around the mask optimization targets, thereby constraining their applicability to practical mask optimization. To advance deep learning for cell- and context-aware mask optimization, we present MaskOpt, a large-scale benchmark dataset constructed from real IC designs at the 45$\mathrm{nm}$ node. MaskOpt includes 104,714 metal-layer tiles and 121,952 via-layer tiles. Each tile is clipped at a standard-cell placement to preserve cell information, exploiting repeated logic gate occurrences. Different context window sizes are supported in MaskOpt to capture the influence of neighboring shapes from optical proximity effects. We evaluate state-of-the-art deep learning models for IC mask optimization to build up benchmarks, and the evaluation results expose distinct trade-offs across baseline models. Further context size analysis and input ablation studies confirm the importance of both surrounding geometries and cell-aware inputs in achieving accurate mask generation.

Method: Proposes a Dual-State Architecture separating workflow state (deterministic control flow) from environment state (stochastic generation). Uses Atomic Action Pairs that couple generation with verification as indivisible transactions, with Guard Functions acting as sensing actions to project probabilistic outputs onto observable workflow state.

Result: Validated on three code generation tasks across 13 LLMs (1.3B-15B parameters). For qualified instruction-following models, task success rates improved by up to 66 percentage points at 1.2-2.1× baseline computational cost.

Conclusion: Architectural constraints can substitute for parameter scale in achieving reliable code generation. Treating LLMs as environment components rather than decision-makers preserves creative stochasticity while enabling deterministic control frameworks.

Abstract: Current approaches to AI coding agents appear to blur the lines between the Large Language Model (LLM) and the agent itself, asking the LLM to make decisions best left to deterministic processes. This leads to systems prone to stochastic failures such as gaming unit tests or hallucinating syntax. Drawing on established software engineering practices that provide deterministic frameworks for managing unpredictable processes, this paper proposes setting the control boundary such that the LLM is treated as a component of the environment environment – preserving its creative stochasticity – rather than the decision-making agent. A \textbf{Dual-State Architecture} is formalized, separating workflow state (deterministic control flow) from environment state (stochastic generation). \textbf{Atomic Action Pairs} couple generation with verification as indivisible transactions, where \textbf{Guard Functions} act as sensing actions that project probabilistic outputs onto observable workflow state. The framework is validated on three code generation tasks across 13 LLMs (1.3B–15B parameters). For qualified instruction-following models, task success rates improved by up to 66 percentage points at 1.2–2.1$\times$ baseline computational cost. The results suggest that architectural constraints can substitute for parameter scale in achieving reliable code generation.

Method: Introduced DominanceBench to systematically analyze the imbalance. Conducted experiments examining data limitations and architectural factors. Analyzed cross-attention dynamics across diffusion timesteps and performed head ablation studies.

Result: Found that limited instance diversity in training data exacerbates inter-concept interference. Dominant tokens rapidly saturate attention, progressively suppressing others across timesteps. DvD behavior arises from distributed attention mechanisms across multiple heads.

Conclusion: The findings provide key insights into generative collapse mechanisms, advancing toward more reliable and controllable text-to-image generation by understanding and addressing the Dominant-vs-Dominated imbalance.

Abstract: Text-to-image diffusion models have drawn significant attention for their ability to generate diverse and high-fidelity images. However, when generating from multi-concept prompts, one concept token often dominates the generation, suppressing the others-a phenomenon we term the Dominant-vs-Dominated (DvD) imbalance. To systematically analyze this imbalance, we introduce DominanceBench and examine its causes from both data and architectural perspectives. Through various experiments, we show that the limited instance diversity in training data exacerbates the inter-concept interference. Analysis of cross-attention dynamics further reveals that dominant tokens rapidly saturate attention, progressively suppressing others across diffusion timesteps. In addition, head ablation studies show that the DvD behavior arises from distributed attention mechanisms across multiple heads. Our findings provide key insights into generative collapse, advancing toward more reliable and controllable text-to-image generation.

Method: The authors first analyze limitations of existing forward-only approaches, then design a forward-only algorithm equivalent to backpropagation under linear/orthogonal assumptions. They relax the linear assumption to create FOTON (Forward-Only Training of Orthogonal Networks).

Result: FOTON outperforms PEPITA and enables training neural networks of any depth without backward passes. It also shows promising results on convolutional networks.

Conclusion: FOTON provides a viable alternative to backpropagation that scales to deep networks and opens avenues for application to advanced architectures like CNNs.

Abstract: Backpropagation is still the de facto algorithm used today to train neural networks. With the exponential growth of recent architectures, the computational cost of this algorithm also becomes a burden. The recent PEPITA and forward-only frameworks have proposed promising alternatives, but they failed to scale up to a handful of hidden layers, yet limiting their use. In this paper, we first analyze theoretically the main limitations of these approaches. It allows us the design of a forward-only algorithm, which is equivalent to backpropagation under the linear and orthogonal assumptions. By relaxing the linear assumption, we then introduce FOTON (Forward-Only Training of Orthogonal Networks) that bridges the gap with the backpropagation algorithm. Experimental results show that it outperforms PEPITA, enabling us to train neural networks of any depth, without the need for a backward pass. Moreover its performance on convolutional networks clearly opens up avenues for its application to more advanced architectures. The code is open-sourced at https://github.com/p0lcAi/FOTON .

Method: Uses a Conditional Variational Autoencoder (CVAE) architecture to synthesize realistic clinical records that are coherent with real-world observations, aiming to increase dataset size and diversity for better cardiac risk prediction.

Result: The proposed architecture successfully generates coherent and realistic synthetic data, and using this synthetic data improves the accuracy of various classifiers for cardiac risk detection, outperforming state-of-the-art deep learning approaches for synthetic data generation.

Conclusion: The CVAE-based approach effectively addresses data limitations in cardiac rehabilitation databases, enabling better risk prediction models while potentially reducing the need for hazardous diagnostic procedures like exercise stress testing.

Abstract: Cardiac rehabilitation constitutes a structured clinical process involving multiple interdependent phases, individualized medical decisions, and the coordinated participation of diverse healthcare professionals. This sequential and adaptive nature enables the program to be modeled as a business process, thereby facilitating its analysis. Nevertheless, studies in this context face significant limitations inherent to real-world medical databases: data are often scarce due to both economic costs and the time required for collection; many existing records are not suitable for specific analytical purposes; and, finally, there is a high prevalence of missing values, as not all patients undergo the same diagnostic tests. To address these limitations, this work proposes an architecture based on a Conditional Variational Autoencoder (CVAE) for the synthesis of realistic clinical records that are coherent with real-world observations. The primary objective is to increase the size and diversity of the available datasets in order to enhance the performance of cardiac risk prediction models and to reduce the need for potentially hazardous diagnostic procedures, such as exercise stress testing. The results demonstrate that the proposed architecture is capable of generating coherent and realistic synthetic data, whose use improves the accuracy of the various classifiers employed for cardiac risk detection, outperforming state-of-the-art deep learning approaches for synthetic data generation.

Method: Dynamic Conflict-Consensus Framework (DCCF) with three key components: 1) Decouples inputs into independent Fact and Sentiment spaces to separate objective mismatches from emotional dissonance; 2) Uses physics-inspired feature dynamics to iteratively polarize representations and extract maximally informative conflicts; 3) Conflict-consensus mechanism that standardizes local discrepancies against global context for robust judgment.

Result: Extensive experiments on three real-world datasets show DCCF consistently outperforms state-of-the-art baselines with an average accuracy improvement of 3.52%.

Conclusion: The inconsistency-seeking paradigm (amplifying contradictions rather than suppressing them) is more effective for fake news detection than traditional consistency-based fusion approaches.

Abstract: Prevalent multimodal fake news detection relies on consistency-based fusion, yet this paradigm fundamentally misinterprets critical cross-modal discrepancies as noise, leading to over-smoothing, which dilutes critical evidence of fabrication. Mainstream consistency-based fusion inherently minimizes feature discrepancies to align modalities, yet this approach fundamentally fails because it inadvertently smoothes out the subtle cross-modal contradictions that serve as the primary evidence of fabrication. To address this, we propose the Dynamic Conflict-Consensus Framework (DCCF), an inconsistency-seeking paradigm designed to amplify rather than suppress contradictions. First, DCCF decouples inputs into independent Fact and Sentiment spaces to distinguish objective mismatches from emotional dissonance. Second, we employ physics-inspired feature dynamics to iteratively polarize these representations, actively extracting maximally informative conflicts. Finally, a conflict-consensus mechanism standardizes these local discrepancies against the global context for robust deliberative judgment.Extensive experiments conducted on three real world datasets demonstrate that DCCF consistently outperforms state-of-the-art baselines, achieving an average accuracy improvement of 3.52%.

Method: HyDRA implements hierarchical optimization (coarse-grained rank assignment to different layers and fine-grained rank adjustment within individual layers) and dynamic adjustment using an end-to-end automatic optimization with a lightweight performance model to determine and adjust ranks during fine-tuning.

Result: HyDRA consistently outperforms baselines, achieving 4.7% improvement across various model sizes without increasing trainable parameters. In some tasks, it even surpasses full-parameter fine-tuning.

Conclusion: HyDRA provides an effective parameter-efficient fine-tuning solution for mobile VLMs through hierarchical and dynamic rank scheduling, addressing computational challenges while maintaining or improving performance.

Abstract: Vision Language Models (VLMs) have undergone significant advancements, particularly with the emergence of mobile-oriented VLMs, which offer a wide range of application scenarios. However, the substantial computational requirements for training these models present a significant obstacle to their practical application. To address this issue, Low-Rank Adaptation (LoRA) has been proposed. Nevertheless, the standard LoRA with a fixed rank lacks sufficient capability for training mobile VLMs that process both text and image modalities. In this work, we introduce HyDRA, a parameter-efficient fine-tuning framework designed to implement hierarchical and dynamic rank scheduling for mobile VLMs. This framework incorporates two essential optimization strategies: (1) hierarchical optimization, which involves a coarse-grained approach that assigns different ranks to various layers, as well as a fine-grained method that adjusts ranks within individual layers, and (2) dynamic adjustment, which employs an end-to-end automatic optimization using a lightweight performance model to determine and adjust ranks during the fine-tuning process. Comprehensive experiments conducted on popular benchmarks demonstrate that HyDRA consistently outperforms the baseline, achieving a 4.7% improvement across various model sizes without increasing the number of trainable parameters. In some tasks, it even surpasses full-parameter fine-tuning.

Method: Evaluated recent VLM-based reward models under a unified framework with identical backbones, finetuning data, and evaluation environments using Meta-World tasks. Assessed modeling accuracy through consistency with ground truth reward and correlation with expert progress.

Result: A simple triplet loss outperformed state-of-the-art methods, suggesting that much of the improvements in recent approaches could be attributed to differences in data and architectures rather than learning objectives.

Conclusion: The learning objective itself may be less critical than previously thought; simpler approaches can achieve comparable or better performance when evaluated under fair, controlled conditions.

Abstract: Learning generalizable reward functions is a core challenge in embodied intelligence. Recent work leverages contrastive vision language models (VLMs) to obtain dense, domain-agnostic rewards without human supervision. These methods adapt VLMs into reward models through increasingly complex learning objectives, yet meaningful comparison remains difficult due to differences in training data, architectures, and evaluation settings. In this work, we isolate the impact of the learning objective by evaluating recent VLM-based reward models under a unified framework with identical backbones, finetuning data, and evaluation environments. Using Meta-World tasks, we assess modeling accuracy by measuring consistency with ground truth reward and correlation with expert progress. Remarkably, we show that a simple triplet loss outperforms state-of-the-art methods, suggesting that much of the improvements in recent approaches could be attributed to differences in data and architectures.

Method: PHOTON maintains a hierarchy of latent streams: a bottom-up encoder compresses tokens into low-rate contextual states, while lightweight top-down decoders reconstruct fine-grained token representations, enabling vertical multi-resolution context access instead of flat scanning.

Result: PHOTON achieves superior throughput-quality trade-off compared to Transformer-based language models, with significant advantages in long-context and multi-query tasks, reducing decode-time KV-cache traffic by up to 1000× higher throughput per unit memory.

Conclusion: The hierarchical architecture of PHOTON addresses fundamental limitations of Transformer’s access pattern, offering a more efficient alternative for autoregressive language modeling with better memory utilization and throughput.

Abstract: Transformers operate as horizontal token-by-token scanners; at each generation step, the model attends to an ever-growing sequence of token-level states. This access pattern increases prefill latency and makes long-context decoding increasingly memory-bound, as KV-cache reads and writes dominate inference throughput rather than arithmetic computation. We propose Parallel Hierarchical Operation for Top-down Networks (PHOTON), a hierarchical autoregressive model that replaces flat scanning with vertical, multi-resolution context access. PHOTON maintains a hierarchy of latent streams: a bottom-up encoder progressively compresses tokens into low-rate contextual states, while lightweight top-down decoders reconstruct fine-grained token representations. Experimental results show that PHOTON is superior to competitive Transformer-based language models regarding the throughput-quality trade-off, offering significant advantages in long-context and multi-query tasks. This reduces decode-time KV-cache traffic, yielding up to $10^{3}\times$ higher throughput per unit memory.

Method: Created FEM-Bench 2025 containing introductory but nontrivial computational mechanics tasks aligned with first graduate course material. Tasks capture essential numerical and physical modeling challenges while representing only a small fraction of discipline complexity. Evaluated state-of-the-art LLMs on function writing and unit test writing capabilities.

Result: Best performing model at function writing (Gemini 3 Pro) completed 30/33 tasks at least once and 26/33 tasks all five times in five-attempt runs. Best performing model at unit test writing (GPT-5) had Average Joint Success Rate of 73.8%. Other popular models showed broad performance variation, indicating current LLMs don’t reliably solve all tasks.

Conclusion: FEM-Bench establishes structured foundation for evaluating AI-generated scientific code. Future iterations will incorporate increasingly sophisticated tasks to track progress as models evolve, addressing the critical need for rigorous benchmarks in physical reasoning evaluation.

Abstract: As LLMs advance their reasoning capabilities about the physical world, the absence of rigorous benchmarks for evaluating their ability to generate scientifically valid physical models has become a critical gap. Computational mechanics, which develops and applies mathematical models and numerical methods to predict the behavior of physical systems under forces, deformation, and constraints, provides an ideal foundation for structured scientific reasoning evaluation. Problems follow clear mathematical structure, enforce strict physical and numerical constraints, and support objective verification. The discipline requires constructing explicit models of physical systems and reasoning about geometry, spatial relationships, and material behavior, connecting directly to emerging AI goals in physical reasoning and world modeling. We introduce FEM-Bench, a computational mechanics benchmark designed to evaluate the ability of LLMs to generate correct finite element method (FEM) and related code. FEM-Bench 2025 contains a suite of introductory but nontrivial tasks aligned with material from a first graduate course on computational mechanics. These tasks capture essential numerical and physical modeling challenges while representing only a small fraction of the complexity present in the discipline. Despite their simplicity, state-of-the-art LLMs do not reliably solve all of them. In a five attempt run, the best performing model at function writing, Gemini 3 Pro, completed 30/33 tasks at least once and 26/33 tasks all five times. The best performing model at unit test writing, GPT-5, had an Average Joint Success Rate of 73.8%. Other popular models showed broad performance variation. FEM-Bench establishes a structured foundation for evaluating AI-generated scientific code, and future iterations will incorporate increasingly sophisticated tasks to track progress as models evolve.

Method: 1) Derive theoretical Lipschitz constants for aggregation methods in multimodal autoencoders. 2) Introduce a regularized attention-based fusion method based on theoretical analysis. 3) Empirically validate findings by estimating Lipschitz constants across multiple trials and fusion strategies.

Result: Proposed fusion function aligns with theoretical predictions and outperforms existing strategies in consistency, convergence speed, and accuracy. Provides empirical validation of theoretical Lipschitz constant estimates.

Conclusion: Work provides solid theoretical foundation for understanding fusion in multimodal autoencoders and contributes a solution for enhancing their performance through Lipschitz analysis and regularized attention-based fusion.

Abstract: In recent years, the development of multimodal autoencoders has gained significant attention due to their potential to handle multimodal complex data types and improve model performance. Understanding the stability and robustness of these models is crucial for optimizing their training, architecture, and real-world applicability. This paper presents an analysis of Lipschitz properties in multimodal autoencoders, combining both theoretical insights and empirical validation to enhance the training stability of these models. We begin by deriving the theoretical Lipschitz constants for aggregation methods within the multimodal autoencoder framework. We then introduce a regularized attention-based fusion method, developed based on our theoretical analysis, which demonstrates improved stability and performance during training. Through a series of experiments, we empirically validate our theoretical findings by estimating the Lipschitz constants across multiple trials and fusion strategies. Our results demonstrate that our proposed fusion function not only aligns with theoretical predictions but also outperforms existing strategies in terms of consistency, convergence speed, and accuracy. This work provides a solid theoretical foundation for understanding fusion in multimodal autoencoders and contributes a solution for enhancing their performance.

Method: Defines a tailored robustness property for early exit architectures, uses off-the-shelf solvers for verification, and presents a baseline algorithm enhanced with early stopping strategy and heuristic optimizations that maintain soundness and completeness.

Result: Experiments on multiple benchmarks validate the framework’s effectiveness. Early exits not only provide natural inference acceleration but also enhance verifiability, enabling more queries to be solved in less time compared to standard networks.

Conclusion: The work demonstrates how early exit architectures can improve both efficiency and verifiability, and provides metrics to help users navigate the trade-off between accuracy and efficiency in AI systems.

Abstract: Ensuring the safety and efficiency of AI systems is a central goal of modern research. Formal verification provides guarantees of neural network robustness, while early exits improve inference efficiency by enabling intermediate predictions. Yet verifying networks with early exits introduces new challenges due to their conditional execution paths. In this work, we define a robustness property tailored to early exit architectures and show how off-the-shelf solvers can be used to assess it. We present a baseline algorithm, enhanced with an early stopping strategy and heuristic optimizations that maintain soundness and completeness. Experiments on multiple benchmarks validate our framework’s effectiveness and demonstrate the performance gains of the improved algorithm. Alongside the natural inference acceleration provided by early exits, we show that they also enhance verifiability, enabling more queries to be solved in less time compared to standard networks. Together with a robustness analysis, we show how these metrics can help users navigate the inherent trade-off between accuracy and efficiency.

Method: Construct datasets of causal graphs and queries spanning difficulty axes (query level: associational, interventional, counterfactual; structural complexity). Fine-tune Qwen-2.5-Instruct models (3B-32B) using RLVR vs SFT, varying model scale and training query levels.

Result: RLVR yields stronger within-level and across-level generalization than SFT, but only for specific combinations of model size and training query level. RLVR improves marginalization strategy and reduces intermediate probability calculation errors when model has sufficient initial competence.

Conclusion: RLVR can improve specific causal reasoning subskills in LLMs, but its benefits emerge only when the model has sufficient initial competence, showing RLVR’s effectiveness depends on the model’s initial reasoning capabilities.

Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising paradigm for post-training large language models (LLMs) on complex reasoning tasks. Yet, the conditions under which RLVR yields robust generalization remain poorly understood. This paper provides an empirical study of RLVR generalization in the setting of probabilistic inference over causal graphical models. This setting offers two natural axes along which to examine generalization: (i) the level of the probabilistic query – associational, interventional, or counterfactual – and (ii) the structural complexity of the query, measured by the size of its relevant subgraph. We construct datasets of causal graphs and queries spanning these difficulty axes and fine-tune Qwen-2.5-Instruct models using RLVR or supervised fine-tuning (SFT). We vary both the model scale (3B-32B) and the query level included in training. We find that RLVR yields stronger within-level and across-level generalization than SFT, but only for specific combinations of model size and training query level. Further analysis shows that RLVR’s effectiveness depends on the model’s initial reasoning competence. With sufficient initial competence, RLVR improves an LLM’s marginalization strategy and reduces errors in intermediate probability calculations, producing substantial accuracy gains, particularly on more complex queries. These findings show that RLVR can improve specific causal reasoning subskills, with its benefits emerging only when the model has sufficient initial competence.

Method: TS-Arena treats the genuinely unknown future as the definitive test environment, implementing pre-registration mechanism on live data streams to ensure evaluation targets remain physically non-existent during inference, enforcing strict global temporal split.

Result: The platform establishes a moving temporal frontier that prevents historical contamination and provides authentic assessment of model generalization, initially applied within the energy sector.

Conclusion: TS-Arena restores operational integrity of forecasting by creating sustainable infrastructure for comparing foundation models under real-world constraints, with prototype available on Hugging Face.

Abstract: While Time Series Foundation Models (TSFMs) offer transformative capabilities for forecasting, they simultaneously risk triggering a fundamental evaluation crisis. This crisis is driven by information leakage due to overlapping training and test sets across different models, as well as the illegitimate transfer of global patterns to test data. While the ability to learn shared temporal dynamics represents a primary strength of these models, their evaluation on historical archives often permits the exploitation of observed global shocks, which violates the independence required for valid benchmarking. We introduce TS-Arena, a platform that restores the operational integrity of forecasting by treating the genuinely unknown future as the definitive test environment. By implementing a pre-registration mechanism on live data streams, the platform ensures that evaluation targets remain physically non-existent during inference, thereby enforcing a strict global temporal split. This methodology establishes a moving temporal frontier that prevents historical contamination and provides an authentic assessment of model generalization. Initially applied within the energy sector, TS-Arena provides a sustainable infrastructure for comparing foundation models under real-world constraints. A prototype of the platform is available at https://huggingface.co/spaces/DAG-UPB/TS-Arena.

Method: Introduces expected prediction entropy (EPE) for evaluating survival models predicting hazard functions, and conditional rank statistics (CRS) for quantifying individual deviations from subgroup survival distributions. Develops eight algorithms including main algorithm combining EPE and CRS.

Result: Theoretical correctness results for main algorithm in well-specified settings. Empirical evaluation shows recovery of ground-truth subgroups in synthetic data and better model fit than naive Cox fitting on whole dataset. Case study on NASA jet engine data reveals known nonlinearities and validates practical design choices.

Conclusion: First comprehensive approach to subgroup discovery for Cox models, with novel metrics and algorithms that successfully identify interpretable subgroups where survival models perform accurately, validated theoretically and empirically across synthetic and real-world applications.

Abstract: We study the problem of subgroup discovery for survival analysis, where the goal is to find an interpretable subset of the data on which a Cox model is highly accurate. Our work is the first to study this particular subgroup problem, for which we make several contributions. Subgroup discovery methods generally require a “quality function” in order to sift through and select the most advantageous subgroups. We first examine why existing natural choices for quality functions are insufficient to solve the subgroup discovery problem for the Cox model. To address the shortcomings of existing metrics, we introduce two technical innovations: the expected prediction entropy (EPE), a novel metric for evaluating survival models which predict a hazard function; and the conditional rank statistics (CRS), a statistical object which quantifies the deviation of an individual point to the distribution of survival times in an existing subgroup. We study the EPE and CRS theoretically and show that they can solve many of the problems with existing metrics. We introduce a total of eight algorithms for the Cox subgroup discovery problem. The main algorithm is able to take advantage of both the EPE and the CRS, allowing us to give theoretical correctness results for this algorithm in a well-specified setting. We evaluate all of the proposed methods empirically on both synthetic and real data. The experiments confirm our theory, showing that our contributions allow for the recovery of a ground-truth subgroup in well-specified cases, as well as leading to better model fit compared to naively fitting the Cox model to the whole dataset in practical settings. Lastly, we conduct a case study on jet engine simulation data from NASA. The discovered subgroups uncover known nonlinearities/homogeneity in the data, and which suggest design choices which have been mirrored in practice.

Method: Developed an optimized Taylor-based algorithm for matrix exponential with rigorous error analysis and dynamic selection strategy for Taylor order and scaling factor to minimize computational effort under prescribed error tolerance.

Result: Extensive numerical experiments show significant acceleration and maintained high numerical stability compared to existing state-of-the-art implementations, establishing the method as highly efficient for large-scale generative modeling.

Conclusion: The proposed Taylor-based algorithm provides an efficient tool for matrix exponential computation in high-throughput generative AI applications, outperforming traditional Padé approximants with scaling and squaring approaches.

Abstract: The matrix exponential is a fundamental operator in scientific computing and system simulation, with applications ranging from control theory and quantum mechanics to modern generative machine learning. While Padé approximants combined with scaling and squaring have long served as the standard, recent Taylor-based methods, which utilize polynomial evaluation schemes that surpass the classical Paterson–Stockmeyer technique, offer superior accuracy and reduced computational complexity. This paper presents an optimized Taylor-based algorithm for the matrix exponential, specifically designed for the high-throughput requirements of generative AI flows. We provide a rigorous error analysis and develop a dynamic selection strategy for the Taylor order and scaling factor to minimize computational effort under a prescribed error tolerance. Extensive numerical experiments demonstrate that our approach provides significant acceleration and maintains high numerical stability compared to existing state-of-the-art implementations. These results establish the proposed method as a highly efficient tool for large-scale generative modeling.

Method: Applied the deep symbolic regression algorithm SEGVAE to determine properties of two-dimensional materials with defects. Compared results with state-of-the-art graph neural network-based methods.

Result: SEGVAE achieved comparable or even identical outcomes to graph neural network methods for predicting properties of 2D materials with defects.

Conclusion: Symbolic regression methods like SEGVAE offer interpretable alternatives to black-box neural networks for scientific applications, demonstrating practical value in materials science research.

Abstract: Machine learning models have become firmly established across all scientific fields. Extracting features from data and making inferences based on them with neural network models often yields high accuracy; however, this approach has several drawbacks. Symbolic regression is a powerful technique for discovering analytical equations that describe data, providing interpretable and generalizable models capable of predicting unseen data. Symbolic regression methods have gained new momentum with the advancement of neural network technologies and offer several advantages, the main one being the interpretability of results. In this work, we examined the application of the deep symbolic regression algorithm SEGVAE to determine the properties of two-dimensional materials with defects. Comparing the results with state-of-the-art graph neural network-based methods shows comparable or, in some cases, even identical outcomes. We also discuss the applicability of this class of methods in natural sciences.

Method: Dual-specialist ensemble framework with two predictive streams: 1) Environmental specialist using Graph Neural Network (GNN) to model community as directed contagion graph with physics-informed weights and Google AlphaEarth embeddings, and 2) Structural specialist using XGBoost to analyze granular asset-level resilience. These are synthesized through logistic stacking.

Result: Applied to 2025 Eaton Fire, the framework revealed neighborhood-scale environmental pressure dominates propagation pathways, while XGBoost identified eaves as primary micro-scale ingress vector. Ensemble achieved robust classification and generated diagnostic risk topology.

Conclusion: The framework enables targeted mitigation strategies: vegetation management for high-connectivity clusters and structural hardening for vulnerable nodes, moving beyond binary loss prediction to operationalize proactive, data-driven community resilience.

Abstract: As wildfires increasingly evolve into urban conflagrations, traditional risk models that treat structures as isolated assets fail to capture the non-linear contagion dynamics characteristic of the wildland urban interface (WUI). This research bridges the gap between mechanistic physics and data driven learning by establishing a novel dual specialist ensemble framework that disentangles vulnerability into two distinct vectors, environmental contagion and structural fragility. The architecture integrates two specialized predictive streams, an environmental specialist, implemented as a graph neural network (GNN) that operationalizes the community as a directed contagion graph weighted by physics informed convection, radiation, and ember probabilities, and enriched with high dimensional Google AlphaEarth Foundation embeddings, and a Structural Specialist, implemented via XGBoost to isolate granular asset level resilience. Applied to the 2025 Eaton Fire, the framework reveals a critical dichotomy in risk drivers. The GNN demonstrates that neighborhood scale environmental pressure overwhelmingly dominates intrinsic structural features in defining propagation pathways, while the XGBoost model identifies eaves as the primary micro scale ingress vector. By synthesizing these divergent signals through logistic stacking, the ensemble achieves robust classification and generates a diagnostic risk topology. This capability empowers decision makers to move beyond binary loss prediction and precisely target mitigation prioritizing vegetation management for high connectivity clusters and structural hardening for architecturally vulnerable nodes thereby operationalizing a proactive, data driven approach to community resilience.

Method: FedMPDD encodes each client’s gradient by computing directional derivatives along multiple random vectors, compressing the gradient from dimension d to m (m « d). The server decodes aggregated information by projecting back onto the same random vectors. Using multiple projections overcomes dimension-dependent convergence limitations of single projections.

Result: Theoretical analysis shows FedMPDD converges at O(1/√K) rate, matching FedSGD performance. Experiments on benchmark datasets validate the theory. The method provides inherent privacy against gradient inversion attacks due to low-rank projection geometry, with tunable privacy-utility trade-off controlled by projection count.

Conclusion: FedMPDD simultaneously optimizes bandwidth utilization and enhances privacy in federated learning through multi-projected directional derivatives, achieving communication efficiency (O(m) vs O(d)) while maintaining convergence performance and providing inherent privacy protection.

Abstract: This paper introduces \texttt{FedMPDD} (\textbf{Fed}erated Learning via \textbf{M}ulti-\textbf{P}rojected \textbf{D}irectional \textbf{D}erivatives), a novel algorithm that simultaneously optimizes bandwidth utilization and enhances privacy in Federated Learning. The core idea of \texttt{FedMPDD} is to encode each client’s high-dimensional gradient by computing its directional derivatives along multiple random vectors. This compresses the gradient into a much smaller message, significantly reducing uplink communication costs from $\mathcal{O}(d)$ to $\mathcal{O}(m)$, where $m \ll d$. The server then decodes the aggregated information by projecting it back onto the same random vectors. Our key insight is that averaging multiple projections overcomes the dimension-dependent convergence limitations of a single projection. We provide a rigorous theoretical analysis, establishing that \texttt{FedMPDD} converges at a rate of $\mathcal{O}(1/\sqrt{K})$, matching the performance of FedSGD. Furthermore, we demonstrate that our method provides some inherent privacy against gradient inversion attacks due to the geometric properties of low-rank projections, offering a tunable privacy-utility trade-off controlled by the number of projections. Extensive experiments on benchmark datasets validate our theory and demonstrates our results.

Method: Uses adversarial training module within mixture-of-experts architecture with nine pre-trained ResNet-18 experts. Jointly updates expert parameters and gating mechanism during end-to-end training for further optimization.

Result: Outperforms state-of-the-art defense systems and plain classifiers, even when those classifiers use more complex architectures than the proposed model’s backbone.

Conclusion: The proposed mixture-of-experts defense system with adversarial training effectively enhances robustness against adversarial threats and achieves superior performance compared to existing approaches.

Abstract: Machine learning is a powerful tool enabling full automation of a huge number of tasks without explicit programming. Despite recent progress of machine learning in different domains, these models have shown vulnerabilities when they are exposed to adversarial threats. Adversarial threats aim to hinder the machine learning models from satisfying their objectives. They can create adversarial perturbations, which are imperceptible to humans’ eyes but have the ability to cause misclassification during inference. Moreover, they can poison the training data to harm the model’s performance or they can query the model to steal its sensitive information. In this paper, we propose a defense system, which devises an adversarial training module within mixture-of-experts architecture to enhance its robustness against adversarial threats. In our proposed defense system, we use nine pre-trained experts with ResNet-18 as their backbone. During end-to-end training, the parameters of expert models and gating mechanism are jointly updated allowing further optimization of the experts. Our proposed defense system outperforms state-of-the-art defense systems and plain classifiers, which use a more complex architecture than our model’s backbone.

Method: The authors use roofline analysis to identify memory bottlenecks in BLR methods for multi-token inference. They introduce custom Triton kernels with partial fusion and memory layout optimizations specifically designed for Monarch and BLAST compression techniques to overcome these memory constraints.

Result: On memory-constrained NVIDIA GPUs (Jetson Orin Nano and A40), the optimized kernels achieve up to 3.76× speedups and 3× model size compression compared to PyTorch dense baselines with CUDA backend and compiler-level optimizations. The approach supports various models including Llama-7/1B, GPT2-S, DiT-XL/2, and ViT-B.

Conclusion: Custom memory-optimized kernels are essential for realizing the full potential of BLR compression methods in practice, especially for multi-token inference scenarios where memory bottlenecks limit performance. The proposed optimizations enable efficient deployment of compressed transformer models on resource-constrained hardware.

Abstract: Recent advances in transformer-based foundation models have made them the default choice for many tasks, but their rapidly growing size makes fitting a full model on a single GPU increasingly difficult and their computational cost prohibitive. Block low-rank (BLR) compression techniques address this challenge by learning compact representations of weight matrices. While traditional low-rank (LR) methods often incur sharp accuracy drops, BLR approaches such as Monarch and BLAST can better capture the underlying structure, thus preserving accuracy while reducing computations and memory footprints. In this work, we use roofline analysis to show that, although BLR methods achieve theoretical savings and practical speedups for single-token inference, multi-token inference often becomes memory-bound in practice, increasing latency despite compiler-level optimizations in PyTorch. To address this, we introduce custom Triton kernels with partial fusion and memory layout optimizations for both Monarch and BLAST. On memory-constrained NVIDIA GPUs such as Jetson Orin Nano and A40, our kernels deliver up to $3.76\times$ speedups and $3\times$ model size compression over PyTorch dense baselines using CUDA backend and compiler-level optimizations, while supporting various models including Llama-7/1B, GPT2-S, DiT-XL/2, and ViT-B. Our code is available at https://github.com/pabillam/mem-efficient-blr .

Method: Proposes the all-pairs paired method which applies paired analysis to all pairs of LLMs, measures three noise components (prediction noise, data noise, total noise) based on millions of question-level predictions across various evals and settings.

Result: Two key findings: 1) Each eval exhibits characteristic and highly predictable total noise levels across all model pairs; 2) Paired prediction noise typically exceeds paired data noise, meaning reducing prediction noise through averaging can significantly increase statistical power.

Conclusion: The findings enable practitioners to assess significance without custom testing and detect much smaller effects in controlled experiments, providing practical tools for more statistically powerful LLM evaluations.

Abstract: Separating signal from noise is central to experimental science. Applying well-established statistical method effectively to LLM evals requires consideration of their unique noise characteristics. We clearly define and measure three types of noise: prediction noise from generating different answers on a given question, data noise from sampling questions, and their combined total noise following the law of total variance. To emphasize relative comparisons and gain statistical power, we propose the all-pairs paired method, which applies the paired analysis to all pairs of LLMs and measures all the noise components based on millions of question-level predictions across many evals and settings. These measurements revealed clear patterns. First, each eval exhibits a characteristic and highly predictable total noise level across all model pairs. Second, paired prediction noise typically exceeds paired data noise, which means reducing prediction noise by averaging can significantly increase statistical power. These findings enable practitioners to assess significance without custom testing and to detect much smaller effects in controlled experiments.

Method: Models gradient-based training as a discrete-time dynamical system and formulates poisoning robustness as formal safety verification. Uses barrier certificates from control theory with sufficient conditions to certify robust radius against worst-case ℓp-norm poisoning. Parameterizes barrier certificates as neural networks trained on poisoned trajectories and derives PAC bounds via scenario convex program.

Result: Approach certifies non-trivial perturbation budgets on MNIST, SVHN, and CIFAR-10 datasets. The framework is model-agnostic, requires no prior knowledge of attack or contamination level, and provides the first unified certification for both training-time and test-time attacks.

Conclusion: The paper presents a principled formal robustness certification framework that provides provable guarantees against data poisoning attacks, bridging control theory and machine learning safety with practical applicability across different attack settings.

Abstract: The increasing use of machine learning in safety-critical domains amplifies the risk of adversarial threats, especially data poisoning attacks that corrupt training data to degrade performance or induce unsafe behavior. Most existing defenses lack formal guarantees or rely on restrictive assumptions about the model class, attack type, extent of poisoning, or point-wise certification, limiting their practical reliability. This paper introduces a principled formal robustness certification framework that models gradient-based training as a discrete-time dynamical system (dt-DS) and formulates poisoning robustness as a formal safety verification problem. By adapting the concept of barrier certificates (BCs) from control theory, we introduce sufficient conditions to certify a robust radius ensuring that the terminal model remains safe under worst-case ${\ell}_p$-norm based poisoning. To make this practical, we parameterize BCs as neural networks trained on finite sets of poisoned trajectories. We further derive probably approximately correct (PAC) bounds by solving a scenario convex program (SCP), which yields a confidence lower bound on the certified robustness radius generalizing beyond the training set. Importantly, our framework also extends to certification against test-time attacks, making it the first unified framework to provide formal guarantees in both training and test-time attack settings. Experiments on MNIST, SVHN, and CIFAR-10 show that our approach certifies non-trivial perturbation budgets while being model-agnostic and requiring no prior knowledge of the attack or contamination level.

Method: Three-level approach: 1) heterogeneous GNN baseline with attention-based message passing, 2) FlowKANet replacing MLP layers with KANs using KAMP-Attn (Kolmogorov-Arnold Message Passing with Attention), 3) distillation into symbolic surrogate models via block-wise regression for closed-form equations.

Result: KAN layers provide favorable trade-off between efficiency and accuracy, and symbolic surrogates enable lightweight deployment while preserving graph-structured dependencies.

Conclusion: FlowKANet with KAN layers offers efficient flow delay prediction, and symbolic distillation enhances transparency and deployment feasibility for network optimization.

Abstract: Accurate prediction of flow delay is essential for optimizing and managing modern communication networks. We investigate three levels of modeling for this task. First, we implement a heterogeneous GNN with attention-based message passing, establishing a strong neural baseline. Second, we propose FlowKANet in which Kolmogorov-Arnold Networks replace standard MLP layers, reducing trainable parameters while maintaining competitive predictive performance. FlowKANet integrates KAMP-Attn (Kolmogorov-Arnold Message Passing with Attention), embedding KAN operators directly into message-passing and attention computation. Finally, we distill the model into symbolic surrogate models using block-wise regression, producing closed-form equations that eliminate trainable weights while preserving graph-structured dependencies. The results show that KAN layers provide a favorable trade-off between efficiency and accuracy and that symbolic surrogates emphasize the potential for lightweight deployment and enhanced transparency.

Method: The thesis proposes time-efficient methods for both red team (vulnerability identification) and blue team (vulnerability mitigation) approaches within the adversarial robustness framework.

Result: The research provides computational efficiency improvements for adversarial robustness evaluation and enhancement, though specific results are not detailed in the abstract.

Conclusion: Time-efficient methods are essential for practical adversarial robustness assessment and improvement in large-scale DNNs, addressing current computational constraints.

Abstract: With deep neural networks (DNNs) increasingly embedded in modern society, ensuring their safety has become a critical and urgent issue. In response, substantial efforts have been dedicated to the red-blue adversarial framework, where the red team focuses on identifying vulnerabilities in DNNs and the blue team on mitigating them. However, existing approaches from both teams remain computationally intensive, constraining their applicability to large-scale models. To overcome this limitation, this thesis endeavours to provide time-efficient methods for the evaluation and enhancement of adversarial robustness in DNNs.

Method: DiEC performs a two-dimensional search over layer × timestep space, decomposed into two stages: 1) Fix U-Net bottleneck as Clustering-friendly Middle Layer (CML), 2) Use Optimal Timestep Search (OTS) to find clustering-optimal timestep t*. Extracts bottleneck features at t* and obtains clustering representations via lightweight residual mapping. Optimizes DEC-style KL self-training objective with adaptive graph regularization and entropy regularization. Includes denoising-consistency branch at random timesteps for stabilization and generative consistency.

Result: DiEC achieves competitive clustering performance on multiple standard benchmarks, demonstrating the effectiveness of leveraging diffusion model internal activations for clustering tasks.

Conclusion: The internal activations of pretrained diffusion models contain valuable clustering information that varies across network hierarchies and noise timesteps. By systematically searching for clustering-friendly representations and incorporating consistency regularization, DiEC enables effective unsupervised clustering directly from diffusion model features without requiring task-specific training.

Abstract: Deep clustering hinges on learning representations that are inherently clusterable. However, using a single encoder to produce a fixed embedding ignores the representation trajectory formed by a pretrained diffusion model across network hierarchies and noise timesteps, where clusterability varies substantially. We propose DiEC (Diffusion Embedded Clustering), which performs unsupervised clustering by directly reading internal activations from a pretrained diffusion U-Net. DiEC formulates representation selection as a two-dimensional search over layer x timestep, and exploits a weak-coupling property to decompose it into two stages. Specifically, we first fix the U-Net bottleneck layer as the Clustering-friendly Middle Layer (CML), and then use Optimal Timestep Search (OTS) to identify the clustering-optimal timestep (t*). During training, we extract bottleneck features at the fixed t* and obtain clustering representations via a lightweight residual mapping. We optimize a DEC-style KL self-training objective, augmented with adaptive graph regularization and entropy regularization to strengthen cluster structures. In parallel, we introduce a denoising-consistency branch at random timesteps to stabilize the representations and preserve generative consistency. Experiments show that DiEC achieves competitive clustering performance on multiple standard benchmarks.

Method: Two-stage framework: (1) Create dataset with problem-agnostic graph features and hardness classifications, train ML classifiers to map features to hardness categories; (2) Use association rule mining (FP-Growth) to explain predictions, plus train regression models to predict computation times. Applied to 3287 maximum clique instances from COLLAB, IMDB, TWITTER datasets using 5 algorithms (Gurobi, CliSAT, MOMC, EGN, HGS).

Result: Excellent prediction performance: weighted F1 score 0.9921, minority-class F1 0.878, ROC-AUC 0.9083 using only 3 graph features. Best association rule had support 0.8829 for hard instances with 87.64% accuracy. Best regression model achieved percentage RMSE 5.12 and R2 0.991 for computation time prediction.

Conclusion: GCO-HPIF effectively predicts and explains computational hardness of combinatorial optimization problems, demonstrating strong performance with minimal features and providing interpretable insights through association rules, making it useful for both prediction and explanation tasks.

Abstract: This study introduces GCO-HPIF, a general machine-learning-based framework to predict and explain the computational hardness of combinatorial optimization problems that can be represented on graphs. The framework consists of two stages. In the first stage, a dataset is created comprising problem-agnostic graph features and hardness classifications of problem instances. Machine-learning-based classification algorithms are trained to map graph features to hardness categories. In the second stage, the framework explains the predictions using an association rule mining algorithm. Additionally, machine-learning-based regression models are trained to predict algorithmic computation times. The GCO-HPIF framework was applied to a dataset of 3287 maximum clique problem instances compiled from the COLLAB, IMDB, and TWITTER graph datasets using five state-of-the-art algorithms, namely three exact branch-and-bound-based algorithms (Gurobi, CliSAT, and MOMC) and two graph-neural-network-based algorithms (EGN and HGS). The framework demonstrated excellent performance in predicting instance hardness, achieving a weighted F1 score of 0.9921, a minority-class F1 score of 0.878, and an ROC-AUC score of 0.9083 using only three graph features. The best association rule found by the FP-Growth algorithm for explaining the hardness predictions had a support of 0.8829 for hard instances and an overall accuracy of 87.64 percent, underscoring the framework’s usefulness for both prediction and explanation. Furthermore, the best-performing regression model for predicting computation times achieved a percentage RMSE of 5.12 and an R2 value of 0.991.

Method: RevFFN introduces a memory-efficient fine-tuning paradigm for mixture of experts (MoE) LLMs using carefully designed reversible Transformer blocks. These blocks allow reconstruction of layer input activations from outputs during backpropagation, eliminating the need to store most intermediate activations in memory.

Result: The approach significantly reduces peak memory consumption for full parameter fine-tuning while preserving the expressive capacity of MoE architectures. This enables efficient full fine-tuning on a single consumer-grade or server-grade GPU.

Conclusion: RevFFN provides a practical solution for memory-efficient full parameter fine-tuning of MoE LLMs, overcoming the memory bottleneck that makes traditional fine-tuning challenging for contemporary large-scale models.

Abstract: Full parameter fine tuning is a key technique for adapting large language models (LLMs) to downstream tasks, but it incurs substantial memory overhead due to the need to cache extensive intermediate activations for backpropagation. This bottleneck makes full fine tuning of contemporary large scale LLMs challenging in practice. Existing distributed training frameworks such as DeepSpeed alleviate this issue using techniques like ZeRO and FSDP, which rely on multi GPU memory or CPU offloading, but often require additional hardware resources and reduce training speed. We introduce RevFFN, a memory efficient fine tuning paradigm for mixture of experts (MoE) LLMs. RevFFN employs carefully designed reversible Transformer blocks that allow reconstruction of layer input activations from outputs during backpropagation, eliminating the need to store most intermediate activations in memory. While preserving the expressive capacity of MoE architectures, this approach significantly reduces peak memory consumption for full parameter fine tuning. As a result, RevFFN enables efficient full fine tuning on a single consumer grade or server grade GPU.

Method: Blends seasonal time-series models with tree-based learners for multivariate demand forecasting, segment-level price elasticity, and churn propensity. Uses Monte Carlo scenario testing to map risk envelopes and solves constrained optimization with business guardrails on customer experience, margin floors, and allowable churn.

Result: Outperforms static tiers and uniform uplifts across heterogeneous SaaS portfolios by reallocating price moves toward segments with higher willingness-to-pay while protecting price-sensitive cohorts. Enables real-time recalibration via modular APIs.

Conclusion: The framework serves as a strategy playbook for transitioning from flat to dynamic pricing, aligning pricing with CLV/MRR targets, and embedding ethical guardrails to enable durable growth without eroding customer trust.

Abstract: This paper presents a marketing analytics framework that operationalizes subscription pricing as a dynamic, guardrailed decision system, uniting multivariate demand forecasting, segment-level price elasticity, and churn propensity to optimize revenue, margin, and retention. The approach blends seasonal time-series models with tree-based learners, runs Monte Carlo scenario tests to map risk envelopes, and solves a constrained optimization that enforces business guardrails on customer experience, margin floors, and allowable churn. Validated across heterogeneous SaaS portfolios, the method consistently outperforms static tiers and uniform uplifts by reallocating price moves toward segments with higher willingness-to-pay while protecting price-sensitive cohorts. The system is designed for real-time recalibration via modular APIs and includes model explainability for governance and compliance. Managerially, the framework functions as a strategy playbook that clarifies when to shift from flat to dynamic pricing, how to align pricing with CLV and MRR targets, and how to embed ethical guardrails, enabling durable growth without eroding customer trust.

Method: Created open-source multi-fidelity aerodynamic dataset for double-delta wings using nested Saltelli sampling, conducted scaling study with MF-VortexNet GNN surrogate using six training datasets (40-1280 snapshots) and models with 0.1-2.4M parameters under fixed training budget.

Result: Test error decreases with data size with power-law exponent of -0.6122, indicating efficient data utilization; optimal sampling density estimated at ~8 samples per dimension in d-dimensional design space; larger models show improved data utilization efficiency.

Conclusion: Established empirical scaling laws for aerodynamic surrogate models, providing guidelines for dataset size requirements and revealing trade-off between dataset generation cost and model training budget.

Abstract: Data-driven surrogate models are increasingly adopted to accelerate vehicle design. However, open-source multi-fidelity datasets and empirical guidelines linking dataset size to model performance remain limited. This study investigates the relationship between training data size and prediction accuracy for a graph neural network (GNN) based surrogate model for aerodynamic field prediction. We release an open-source, multi-fidelity aerodynamic dataset for double-delta wings, comprising 2448 flow snapshots across 272 geometries evaluated at angles of attack from 11 (degree) to 19 (degree) at Ma=0.3 using both Vortex Lattice Method (VLM) and Reynolds-Averaged Navier-Stokes (RANS) solvers. The geometries are generated using a nested Saltelli sampling scheme to support future dataset expansion and variance-based sensitivity analysis. Using this dataset, we conduct a preliminary empirical scaling study of the MF-VortexNet surrogate by constructing six training datasets with sizes ranging from 40 to 1280 snapshots and training models with 0.1 to 2.4 million parameters under a fixed training budget. We find that the test error decreases with data size with a power-law exponent of -0.6122, indicating efficient data utilization. Based on this scaling law, we estimate that the optimal sampling density is approximately eight samples per dimension in a d-dimensional design space. The results also suggest improved data utilization efficiency for larger surrogate models, implying a potential trade-off between dataset generation cost and model training budget.

Method: Uses Gaussian process operator learning to construct functional representations directly on function space, independent of specific equations or discretizations. Incorporates physical priors through prior mean or kernel design.

Result: Demonstrated on Wetterich and Wilson-Polchinski equations, achieving equal or better performance than existing approximations like local-potential approximation. Method can handle non-constant fields, making it suitable for complex field configurations like instantons.

Conclusion: The Gaussian process operator learning framework provides a flexible, powerful approach for solving functional renormalization group equations, offering superior performance and the ability to handle complex field configurations beyond the capabilities of traditional approximations.

Abstract: We present an operator learning framework for solving non-perturbative functional renormalization group equations, which are integro-differential equations defined on functionals. Our proposed approach uses Gaussian process operator learning to construct a flexible functional representation formulated directly on function space, making it independent of a particular equation or discretization. Our method is flexible, and can apply to a broad range of functional differential equations while still allowing for the incorporation of physical priors in either the prior mean or the kernel design. We demonstrate the performance of our method on several relevant equations, such as the Wetterich and Wilson–Polchinski equations, showing that it achieves equal or better performance than existing approximations such as the local-potential approximation, while being significantly more flexible. In particular, our method can handle non-constant fields, making it promising for the study of more complex field configurations, such as instantons.

Method: A target-agnostic framework using ESM-2 protein embeddings to identify similar proteins from PDB, decomposing known drug ligands into fragments to initialize search space, and employing PPO agents with ChemBERTa-encoded molecules and reaction-template-based transformations for chemically valid molecular generation.

Result: Generates de novo drug candidates with competitive binding affinities and high synthetic accessibility, ensuring 100% chemical validity and novelty as per MOSES benchmarking standards.

Conclusion: Demonstrates the potential of integrating structural biology, deep representation learning, and chemical synthesis rules to automate and accelerate rational drug design through a fully integrated RL framework.

Abstract: De novo drug design is a crucial component of modern drug development, yet navigating the vast chemical space to find synthetically accessible, high-affinity candidates remains a significant challenge. Reinforcement Learning (RL) enhances this process by enabling multi-objective optimization and exploration of novel chemical space - capabilities that traditional supervised learning methods lack. In this work, we introduce \textbf{ReACT-Drug}, a fully integrated, target-agnostic molecular design framework based on Reinforcement Learning. Unlike models requiring target-specific fine-tuning, ReACT-Drug utilizes a generalist approach by leveraging ESM-2 protein embeddings to identify similar proteins for a given target from a knowledge base such as Protein Data Base (PDB). Thereafter, the known drug ligands corresponding to such proteins are decomposed to initialize a fragment-based search space, biasing the agent towards biologically relevant subspaces. For each such fragment, the pipeline employs a Proximal Policy Optimization (PPO) agent guiding a ChemBERTa-encoded molecule through a dynamic action space of chemically valid, reaction-template-based transformations. This results in the generation of \textit{de novo} drug candidates with competitive binding affinities and high synthetic accessibility, while ensuring 100% chemical validity and novelty as per MOSES benchmarking. This architecture highlights the potential of integrating structural biology, deep representation learning, and chemical synthesis rules to automate and accelerate rational drug design. The dataset and code are available at https://github.com/YadunandanRaman/ReACT-Drug/.

Method: Designed a prediction task where crucial latent variables are hidden in image data rather than tabular features. Used a synthetic property insurance dataset to test agentic AI systems that generate generic code for tabular modeling versus approaches that incorporate domain-specific insights.

Result: Agentic AI systems relying on generic analytics workflows performed poorly compared to methods using domain-specific insights. Human experts could identify important hidden variables using domain knowledge, while generic AI systems could not.

Conclusion: Current agentic AI for data science has a key limitation in recognizing and incorporating domain knowledge, especially when crucial information is embedded in non-tabular data. Future research should focus on developing AI systems that can better identify and utilize domain-specific insights.

Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous domains. Recent developments in large language models (LLMs) have significantly automated data science workflows, but a fundamental question persists: Can these agentic AI systems truly match the performance of human data scientists who routinely leverage domain-specific knowledge? We explore this question by designing a prediction task where a crucial latent variable is hidden in relevant image data instead of tabular features. As a result, agentic AI that generates generic codes for modeling tabular data cannot perform well, while human experts could identify the important hidden variable using domain knowledge. We demonstrate this idea with a synthetic dataset for property insurance. Our experiments show that agentic AI that relies on generic analytics workflow falls short of methods that use domain-specific insights. This highlights a key limitation of the current agentic AI for data science and underscores the need for future research to develop agentic AI systems that can better recognize and incorporate domain knowledge.

Method: Theoretical analysis using a two-layer ReLU denoising autoencoder (DAE) to study representation structures, plus empirical validation on real-world unconditional and text-to-image diffusion models. Proposes representation-based memorization detection and training-free editing via representation steering.

Result: Memorization corresponds to storing raw training samples in weights with localized “spiky” representations, while generalization captures local data statistics with “balanced” representations. These patterns hold in deep generative models, enabling practical detection and editing techniques.

Conclusion: Learning good representations is central to novel and meaningful generative modeling. The representation-based approach provides tools to detect memorization and control generation through representation steering.

Abstract: Diffusion models excel at generating high-quality, diverse samples, yet they risk memorizing training data when overfit to the training objective. We analyze the distinctions between memorization and generalization in diffusion models through the lens of representation learning. By investigating a two-layer ReLU denoising autoencoder (DAE), we prove that (i) memorization corresponds to the model storing raw training samples in the learned weights for encoding and decoding, yielding localized “spiky” representations, whereas (ii) generalization arises when the model captures local data statistics, producing “balanced” representations. Furthermore, we validate these theoretical findings on real-world unconditional and text-to-image diffusion models, demonstrating that the same representation structures emerge in deep generative models with significant practical implications. Building on these insights, we propose a representation-based method for detecting memorization and a training-free editing technique that allows precise control via representation steering. Together, our results highlight that learning good representations is central to novel and meaningful generative modeling.

Method: GLiBRL uses generalized linear models with learnable basis functions to enable efficient learning of transition and reward models. It provides fully tractable marginal likelihood and Bayesian inference on task parameters and model noises.

Result: On MetaWorld ML10/45 benchmarks, GLiBRL improves VariBAD’s success rate by up to 2.7x. It consistently demonstrates low-variance and decent performance compared to MAML, RL2, SDVT, TrMRL, and ECET.

Conclusion: GLiBRL addresses limitations of existing deep BRL methods by providing tractable Bayesian inference and efficient model learning, achieving superior performance on challenging meta-RL benchmarks.

Abstract: Bayesian Reinforcement Learning (BRL) provides a framework for generalisation of Reinforcement Learning (RL) problems from its use of Bayesian task parameters in the transition and reward models. However, classical BRL methods assume known forms of transition and reward models, reducing their applicability in real-world problems. As a result, recent deep BRL methods have started to incorporate model learning, though the use of neural networks directly on the joint data and task parameters requires optimising the Evidence Lower Bound (ELBO). ELBOs are difficult to optimise and may result in indistinctive task parameters, hence compromised BRL policies. To this end, we introduce a novel deep BRL method, Generalised Linear Models in Deep Bayesian RL with Learnable Basis Functions (GLiBRL), that enables efficient and accurate learning of transition and reward models, with fully tractable marginal likelihood and Bayesian inference on task parameters and model noises. On challenging MetaWorld ML10/45 benchmarks, GLiBRL improves the success rate of one of the state-of-the-art deep BRL methods, VariBAD, by up to 2.7x. Comparing against representative or recent deep BRL / Meta-RL methods, such as MAML, RL2, SDVT, TrMRL and ECET, GLiBRL also demonstrates its low-variance and decent performance consistently.

Method: CoSeNet uses a four-layer architecture (input, formatting, re-scaling, segmentation) with overlapping techniques and pre-trained ML algorithms. It optimizes re-scaling layer parameters using a heuristic algorithm with Window Difference-based fitness metric.

Result: CoSeNet effectively identifies correlated segments better than previous approaches, producing binary noise-free matrices with optimal segmentation points, offering trade-offs between efficiency, memory, and speed.

Conclusion: CoSeNet provides a robust and generalizable solution for correlation segmentation that outperforms existing methods and can be applied to various real-world applications requiring optimal segmentation of correlated data.

Abstract: In this paper, we propose a novel approach for the optimal identification of correlated segments in noisy correlation matrices. The proposed model is known as CoSeNet (Correlation Seg-mentation Network) and is based on a four-layer algorithmic architecture that includes several processing layers: input, formatting, re-scaling, and segmentation layer. The proposed model can effectively identify correlated segments in such matrices, better than previous approaches for similar problems. Internally, the proposed model utilizes an overlapping technique and uses pre-trained Machine Learning (ML) algorithms, which makes it robust and generalizable. CoSeNet approach also includes a method that optimizes the parameters of the re-scaling layer using a heuristic algorithm and fitness based on a Window Difference-based metric. The output of the model is a binary noise-free matrix representing optimal segmentation as well as its seg-mentation points and can be used in a variety of applications, obtaining compromise solutions between efficiency, memory, and speed of the proposed deployment model.

Method: Competitive Swiss-System Dynamics (CSD) framework simulates multi-round sequential contests where models are dynamically paired across curated benchmarks based on win-loss records. Uses Monte Carlo Simulation (100,000 iterations) to compute Expected Win Score, and implements Failure Sensitivity Analysis by parameterizing per-round elimination quantity to profile risk appetite.

Result: CSD provides more nuanced and context-aware rankings than traditional aggregate scoring and static pairwise models, distinguishing between robust generalists and aggressive specialists through risk profiling.

Conclusion: CSD represents a vital step towards risk-informed, next-generation LLM evaluation by moving beyond static metrics to capture dynamic competitive fitness and vulnerability in sequential tasks.

Abstract: The rapid proliferation of Large Language Models (LLMs) and diverse specialized benchmarks necessitates a shift from fragmented, task-specific metrics to a holistic, competitive ranking system that effectively aggregates performance across multiple ability dimensions. Primarily using static scoring, current evaluation methods are fundamentally limited. They struggle to determine the proper mix ratio across diverse benchmarks, and critically, they fail to capture a model’s dynamic competitive fitness or its vulnerability when confronted with sequential, high-stakes tasks. To address this, we introduce the novel Competitive Swiss-System Dynamics (CSD) framework. CSD simulates a multi-round, sequential contest where models are dynamically paired across a curated sequence of benchmarks based on their accumulated win-loss record. And Monte Carlo Simulation ($N=100,000$ iterations) is used to approximate the statistically robust Expected Win Score ($E[S_m]$), which eliminates the noise of random pairing and early-round luck. Furthermore, we implement a Failure Sensitivity Analysis by parameterizing the per-round elimination quantity ($T_k$), which allows us to profile models based on their risk appetite–distinguishing between robust generalists and aggressive specialists. We demonstrate that CSD provides a more nuanced and context-aware ranking than traditional aggregate scoring and static pairwise models, representing a vital step towards risk-informed, next-generation LLM evaluation.

Method: Derived Neural Feature Dynamics (NFD) for ResNets with single-layer residual blocks in joint infinite-width/depth limit. Analyzed vanishing mechanism with 1/sqrt(depth) scaling, studied two-layer blocks, and proposed depth-aware learning-rate correction.

Result: NFD identifies when scaling-law trends persist and explains diminishing returns. Shows GIA becomes valid at infinite depth, provides structural explanation for depth-muP failure, and demonstrates learning-rate correction restores hyperparameter transfer in deeper ResNets.

Conclusion: NFD provides rigorous theory for deep feature learning, explains scaling breakdowns, and offers practical solution (depth-aware learning rates) to enable successful scaling of deep ResNets.

Abstract: The empirical success of deep learning is often attributed to scaling laws that predict consistent gains as model, data, and compute grow; however, large models can exhibit training instability and diminishing returns, suggesting that scaling laws describe what success looks like but not when and why scaling succeeds or fails. A central obstacle is the lack of a rigorous understanding of feature learning at large depth. While muP characterizes feature-learning dynamics in the infinite-width limit and enables hyperparameter transfer across width, its depth extension (depth-muP) breaks down for residual blocks with more than one internal layer. We derive Neural Feature Dynamics (NFD) for ResNets with single-layer residual blocks, characterizing feature learning via a coupled forward-backward stochastic system in the joint infinite-width and infinite-depth limit. In this regime, NFD identifies when scaling-law trends persist and explains diminishing returns. It also reveals a vanishing mechanism induced by the 1/sqrt(depth) residual scaling under which the gradient-independence assumption (GIA), known to fail during training at finite depth, becomes provably valid again at infinite depth, yielding an analytically tractable regime for end-to-end feature learning. Motivated by this insight, we study two-layer residual blocks and show that the same mechanism causes feature-learning collapse in the first internal layer at large depth, providing a structural explanation for the empirical failure of depth-muP. Based on this diagnosis, we propose a depth-aware learning-rate correction that counteracts the collapse and empirically restores depth-wise hyperparameter transfer, yielding stronger performance in deeper ResNets.

Method: Builds shared encoding structure for unified representation of monitoring indicators, employs state fusion mechanism for multi-scale trend analysis, introduces cross-task structural propagation module for dependency modeling, and incorporates dynamic adjustment mechanism for non-stationary behavior adaptation.

Result: Superior performance on multiple error metrics compared to other models, with accurate future state representations under different operating conditions, validated through hyperparameter sensitivity, environmental sensitivity, and data sensitivity analyses.

Conclusion: The unified forecasting framework provides reliable predictive capability for high-dimensional, multi-task, strongly dynamic cloud native environments and offers essential technical support for intelligent backend management.

Abstract: This study proposes a unified forecasting framework for high-dimensional multi-task time series to meet the prediction demands of cloud native backend systems operating under highly dynamic loads, coupled metrics, and parallel tasks. The method builds a shared encoding structure to represent diverse monitoring indicators in a unified manner and employs a state fusion mechanism to capture trend changes and local disturbances across different time scales. A cross-task structural propagation module is introduced to model potential dependencies among nodes, enabling the model to understand complex structural patterns formed by resource contention, link interactions, and changes in service topology. To enhance adaptability to non-stationary behaviors, the framework incorporates a dynamic adjustment mechanism that automatically regulates internal feature flows according to system state changes, ensuring stable predictions in the presence of sudden load shifts, topology drift, and resource jitter. The experimental evaluation compares multiple models across various metrics and verifies the effectiveness of the framework through analyses of hyperparameter sensitivity, environmental sensitivity, and data sensitivity. The results show that the proposed method achieves superior performance on several error metrics and provides more accurate representations of future states under different operating conditions. Overall, the unified forecasting framework offers reliable predictive capability for high-dimensional, multi-task, and strongly dynamic environments in cloud native systems and provides essential technical support for intelligent backend management.

Method: Analyze single-layer Transformers from dynamical systems perspective, interpreting causal self-attention as linear history-dependent recurrence. Study through linear and nonlinear case studies to identify operational regimes.

Result: For linear systems: softmax attention’s convexity constraint restricts representable dynamics, causing oversmoothing in oscillatory settings. For nonlinear systems under partial observability: attention acts as adaptive delay-embedding mechanism enabling state reconstruction with sufficient temporal context and latent dimensionality.

Conclusion: The analysis bridges empirical observations with classical dynamical systems theory, providing insight into when and why Transformers succeed or fail as models of dynamical systems, though no new forecasting model is proposed.

Abstract: Transformers are increasingly adopted for modeling and forecasting time-series, yet their internal mechanisms remain poorly understood from a dynamical systems perspective. In contrast to classical autoregressive and state-space models, which benefit from well-established theoretical foundations, Transformer architectures are typically treated as black boxes. This gap becomes particularly relevant as attention-based models are considered for general-purpose or zero-shot forecasting across diverse dynamical regimes. In this work, we do not propose a new forecasting model, but instead investigate the representational capabilities and limitations of single-layer Transformers when applied to dynamical data. Building on a dynamical systems perspective we interpret causal self-attention as a linear, history-dependent recurrence and analyze how it processes temporal information. Through a series of linear and nonlinear case studies, we identify distinct operational regimes. For linear systems, we show that the convexity constraint imposed by softmax attention fundamentally restricts the class of dynamics that can be represented, leading to oversmoothing in oscillatory settings. For nonlinear systems under partial observability, attention instead acts as an adaptive delay-embedding mechanism, enabling effective state reconstruction when sufficient temporal context and latent dimensionality are available. These results help bridge empirical observations with classical dynamical systems theory, providing insight into when and why Transformers succeed or fail as models of dynamical systems.

Method: STLDM uses a two-stage approach: 1) deterministic forecasting handled by a conditioning network, and 2) enhancement stage performed by a latent diffusion model that learns end-to-end latent representations with both VAE and conditioning network.

Result: Experimental results on multiple radar datasets show STLDM achieves superior performance compared to state-of-the-art methods while also improving inference efficiency.

Conclusion: STLDM presents an effective diffusion-based architecture for precipitation nowcasting that addresses the limitations of both deterministic and generative approaches through its two-stage design.

Abstract: Precipitation nowcasting is a critical spatio-temporal prediction task for society to prevent severe damage owing to extreme weather events. Despite the advances in this field, the complex and stochastic nature of this task still poses challenges to existing approaches. Specifically, deterministic models tend to produce blurry predictions while generative models often struggle with poor accuracy. In this paper, we present a simple yet effective model architecture termed STLDM, a diffusion-based model that learns the latent representation from end to end alongside both the Variational Autoencoder and the conditioning network. STLDM decomposes this task into two stages: a deterministic forecasting stage handled by the conditioning network, and an enhancement stage performed by the latent diffusion model. Experimental results on multiple radar datasets demonstrate that STLDM achieves superior performance compared to the state of the art, while also improving inference efficiency. The code is available in https://github.com/sqfoo/stldm_official.

Method: MODE dynamically combines multiple coreset selection strategies based on their evolving contribution to model performance. It adapts selection criteria to different training phases: emphasizing class balance early, diversity during representation learning, and uncertainty at convergence. The framework achieves (1-1/e)-approximation with O(n log n) complexity.

Result: MODE demonstrates competitive accuracy while providing interpretable insights into data utility evolution. Experiments show it reduces memory requirements compared to static methods.

Conclusion: Dynamic adaptation of coreset selection strategies based on training phases is more effective than static methods, achieving better data efficiency, interpretable utility insights, and reduced memory requirements while maintaining competitive model performance.

Abstract: We present Mode(Multi-Objective adaptive Data Efficiency), a framework that dynamically combines coreset selection strategies based on their evolving contribution to model performance. Unlike static methods, \mode adapts selection criteria to training phases: emphasizing class balance early, diversity during representation learning, and uncertainty at convergence. We show that MODE achieves (1-1/e)-approximation with O(n \log n) complexity and demonstrates competitive accuracy while providing interpretable insights into data utility evolution. Experiments show \mode reduces memory requirements

Method: BALLAST uses lightweight online adaptation with contextual bandits, selecting from discrete timeout options using efficient linear contextual bandits (LinUCB variants) with safe exploration to cap risk during unstable periods.

Result: BALLAST substantially reduces recovery time and unwritable time compared to standard randomized timeouts and common heuristics across challenging WAN regimes, while remaining competitive in stable LAN/WAN settings.

Conclusion: Contextual bandits provide an effective alternative to static timeout heuristics for Raft, improving liveness under network variability while maintaining safety through controlled exploration.

Abstract: Randomized election timeouts are a simple and effective liveness heuristic for Raft, but they become brittle under long-tail latency, jitter, and partition recovery, where repeated split votes can inflate unavailability. This paper presents BALLAST, a lightweight online adaptation mechanism that replaces static timeout heuristics with contextual bandits. BALLAST selects from a discrete set of timeout “arms” using efficient linear contextual bandits (LinUCB variants), and augments learning with safe exploration to cap risk during unstable periods. We evaluate BALLAST on a reproducible discrete-event simulation with long-tail delay, loss, correlated bursts, node heterogeneity, and partition/recovery turbulence. Across challenging WAN regimes, BALLAST substantially reduces recovery time and unwritable time compared to standard randomized timeouts and common heuristics, while remaining competitive on stable LAN/WAN settings.

Method: U-GEPSVM extends GEPSVM with Universum constraints via ratio-based objective function; IU-GEPSVM enhances stability with weighted difference-based formulation providing independent control over class separation and Universum alignment.

Result: IU-GEPSVM achieves peak accuracies of 85% (O vs S) and 80% (Z vs S), with mean accuracies of 81.29% and 77.57% respectively, outperforming baseline methods on Bonn University EEG dataset.

Conclusion: The proposed Universum-enhanced classifiers effectively improve EEG classification performance, with IU-GEPSVM showing superior stability and accuracy for seizure detection tasks.

Abstract: The paper presents novel Universum-enhanced classifiers: the Universum Generalized Eigenvalue Proximal Support Vector Machine (U-GEPSVM) and the Improved U-GEPSVM (IU-GEPSVM) for EEG signal classification. Using the computational efficiency of generalized eigenvalue decomposition and the generalization benefits of Universum learning, the proposed models address critical challenges in EEG analysis: non-stationarity, low signal-to-noise ratio, and limited labeled data. U-GEPSVM extends the GEPSVM framework by incorporating Universum constraints through a ratio-based objective function, while IU-GEPSVM enhances stability through a weighted difference-based formulation that provides independent control over class separation and Universum alignment. The models are evaluated on the Bonn University EEG dataset across two binary classification tasks: (O vs S)-healthy (eyes closed) vs seizure, and (Z vs S)-healthy (eyes open) vs seizure. IU-GEPSVM achieves peak accuracies of 85% (O vs S) and 80% (Z vs S), with mean accuracies of 81.29% and 77.57% respectively, outperforming baseline methods.

Method: Introduces ICRL (in-context RL) prompting - a multi-round framework where LLMs receive numerical scalar rewards after each response, then are prompted again with concatenated prior responses and rewards to iteratively improve performance.

Result: ICRL prompting shows significant improvements over baselines (Self-Refine, Reflexion) on Game of 24, creative writing, ScienceWorld, and Olympiad-level math competitions. Even when rewards come from the same LLM, performance still improves.

Conclusion: LLMs can optimize scalar reward signals during inference, exhibiting RL-like behavior, offering a promising paradigm for test-time scaling and self-improvement without additional training.

Abstract: Reinforcement learning (RL) is a framework for solving sequential decision-making problems. In this work, we demonstrate that, surprisingly, RL emerges during the inference time of large language models (LLMs), a phenomenon we term in-context RL (ICRL). To reveal this capability, we introduce a simple multi-round prompting framework, we call ICRL prompting, for inference-time self-improvement. The goal of ICRL prompting is to guide LLMs to perform reinforcement learning during inference for self-improvement on a given task. After each response, the model receives numerical scalar feedback, denoted as a reward. In the next round, we prompt the LLM again together with a context that concatenates all prior responses and their associated rewards. We consistently observe that response quality improves as the context grows. In other words, the LLM can optimize scalar reward signals during inference, exhibiting behavior analogous to reinforcement learning. We evaluate ICRL prompting on Game of 24, creative writing, ScienceWorld, and Olympiad-level math competitions (AIME and HMMT), demonstrating significant improvements over baselines such as Self-Refine and Reflexion. Notably, even when the reward signals are generated by the same LLM, ICRL prompting still improves performance, highlighting a promising new paradigm for test-time scaling.

Method: Introduces Learning Stability Profiles tracking infinitesimal responses to perturbations along learning trajectories. Proves Fundamental Analytic Stability Theorem linking bounded stability signatures to Lyapunov-type energy dissipation. Extends to non-smooth systems using Clarke generalized derivatives and variational Lyapunov functionals.

Result: Establishes equivalence between uniform boundedness of stability signatures and existence of dissipative Lyapunov-type energy. Derives explicit stability exponents linking spectral norms, activation regularity, step sizes, and learning rates to contractivity. Unifies classical stability results for feedforward networks, residual architectures, and stochastic gradient methods.

Conclusion: Provides a unified dynamical framework for stability analysis across architectures and optimization methods, clarifying joint effects of architectural and algorithmic choices on robustness. Serves as foundation for extensions to continuous-time limits and geometric formulations of learning dynamics.

Abstract: We propose a unified analytic and variational framework for studying stability in deep learning systems viewed as coupled representation-parameter dynamics. The central object is the Learning Stability Profile, which tracks the infinitesimal response of representations, parameters, and update mechanisms to perturbations along the learning trajectory. We prove a Fundamental Analytic Stability Theorem showing that uniform boundedness of these stability signatures is equivalent, up to norm equivalence, to the existence of a Lyapunov-type energy that dissipates along the learning flow. In smooth regimes, the framework yields explicit stability exponents linking spectral norms, activation regularity, step sizes, and learning rates to contractivity of the learning dynamics. Classical spectral stability results for feedforward networks, a discrete CFL-type condition for residual architectures, and parametric and temporal stability laws for stochastic gradient methods arise as direct consequences. The theory extends to non-smooth learning systems, including ReLU networks, proximal and projected updates, and stochastic subgradient flows, by replacing classical derivatives with Clarke generalized derivatives and smooth energies with variational Lyapunov functionals. The resulting framework provides a unified dynamical description of stability across architectures and optimization methods, clarifying how architectural and algorithmic choices jointly govern robustness and sensitivity to perturbations. It also provides a foundation for further extensions to continuous-time limits and geometric formulations of learning dynamics.

Method: Proposes MiST (mid-stage scientific training): data-mixing with SMILES/CIF-aware pre-processing, continued pre-training on 2.9B tokens, and supervised fine-tuning on 1B tokens. These techniques aim to satisfy two identified prerequisites: symbolic competence and latent chemical knowledge.

Result: MiST raises latent-solvability score on 3B and 7B models by up to 1.8x. RL then lifts top-1 accuracy from 10.9% to 63.9% on organic reaction naming, and from 40.6% to 67.4% on inorganic material generation. Similar improvements on other chemical tasks with interpretable reasoning traces.

Conclusion: The paper defines clear prerequisites for chemical reasoning training (symbolic competence and latent chemical knowledge) and demonstrates the importance of mid-stage training in unlocking reasoning capabilities in LLMs for scientific domains.

Abstract: Large Language Models can develop reasoning capabilities through online fine-tuning with rule-based rewards. However, recent studies reveal a critical constraint: reinforcement learning succeeds only when the base model already assigns non-negligible probability to correct answers – a property we term ’latent solvability’. This work investigates the emergence of chemical reasoning capabilities and what these prerequisites mean for chemistry. We identify two necessary conditions for RL-based chemical reasoning: 1) Symbolic competence, and 2) Latent chemical knowledge. We propose mid-stage scientific training (MiST): a set of mid-stage training techniques to satisfy these, including data-mixing with SMILES/CIF-aware pre-processing, continued pre-training on 2.9B tokens, and supervised fine-tuning on 1B tokens. These steps raise the latent-solvability score on 3B and 7B models by up to 1.8x, and enable RL to lift top-1 accuracy from 10.9 to 63.9% on organic reaction naming, and from 40.6 to 67.4% on inorganic material generation. Similar results are observed for other challenging chemical tasks, while producing interpretable reasoning traces. Our results define clear prerequisites for chemical reasoning training and highlight the broader role of mid-stage training in unlocking reasoning capabilities.

Method: Proposes ARS-OPT, a momentum-based algorithm inspired by Nesterov’s Accelerated Gradient (NAG) that proactively estimates gradients with respect to future ray directions using accumulated momentum. Also introduces PARS-OPT which incorporates surrogate-model priors for enhanced gradient estimation.

Result: Theoretical analysis shows ARS-OPT enables more accurate directional updates and achieves faster, more stable optimization. Extensive experiments on ImageNet and CIFAR-10 demonstrate superiority over 13 state-of-the-art approaches in query efficiency.

Conclusion: The proposed momentum-based optimization approach significantly improves query efficiency in hard-label black-box adversarial attacks, making such attacks more practical for real-world deployment with theoretical guarantees and empirical validation.

Abstract: In hard-label black-box adversarial attacks, where only the top-1 predicted label is accessible, the prohibitive query complexity poses a major obstacle to practical deployment. In this paper, we focus on optimizing a representative class of attacks that search for the optimal ray direction yielding the minimum $\ell_2$-norm perturbation required to move a benign image into the adversarial region. Inspired by Nesterov’s Accelerated Gradient (NAG), we propose a momentum-based algorithm, ARS-OPT, which proactively estimates the gradient with respect to a future ray direction inferred from accumulated momentum. We provide a theoretical analysis of its convergence behavior, showing that ARS-OPT enables more accurate directional updates and achieves faster, more stable optimization. To further accelerate convergence, we incorporate surrogate-model priors into ARS-OPT’s gradient estimation, resulting in PARS-OPT with enhanced performance. The superiority of our approach is supported by theoretical guarantees under standard assumptions. Extensive experiments on ImageNet and CIFAR-10 demonstrate that our method surpasses 13 state-of-the-art approaches in query efficiency.

Method: 1) Establish flatness-aware PAC-Bayes generalization bound with “cross-task heterogeneity” term; 2) Frame merging as multi-teacher knowledge distillation on unlabeled data; 3) Implement SAMerging using Sharpness-Aware Minimization to find flat minima.

Result: SAMerging achieves state-of-the-art performance across vision and NLP benchmarks, demonstrating remarkable empirical performance improvements over existing methods.

Conclusion: Theoretical analysis provides principled foundation for model merging, and SAMerging operationalizes these insights to achieve superior generalization through flat minima optimization.

Abstract: Model merging has emerged as a lightweight alternative to joint multi-task learning (MTL), yet the generalization properties of merged models remain largely unexplored. Establishing such theoretical guarantees is non-trivial, as the merging process typically forbids access to the original training data and involves combining fine-tuned models trained on fundamentally heterogeneous data distributions. Without a principled understanding of these dynamics, current methods often rely on heuristics to approximate the optimal combination of parameters. This dependence is most critical in coefficient scaling, the weighting factors that modulate the magnitude of each fine-tuned model’s contribution to the shared parameter. However, without a principled objective to guide their selection, these methods lead to brittle performance and are highly sensitive to scaling initialization. We address this gap by (i) establishing a novel flatness-aware PAC-Bayes generalization bound specifically for the model merging setting. This analysis introduces a “cross-task heterogeneity” term that formally captures the mismatch between diverse fine-tuned model priors and the target multi-task distributions. Guided by this theoretical insight, (ii) we frame model merging as multi-teacher knowledge distillation on scarce, unlabeled data. We formally demonstrate that minimizing the student-teacher Kullback-Leibler divergence directly tightens the upper bound on the merged model’s excess risk. Guided by the flatness-aware bound derived, (iii) we operationalize this objective via SAMerging, a method that employs Sharpness-Aware Minimization (SAM) to find flat minima. Empirically, SAMerging establishes a new state of the art across vision and NLP benchmarks, achieving remarkable performance. The code is available at https://github.com/arshandalili/SAMerging.

Method: TFPI introduces a ThinkFree operation that explicitly discards thinking content via direct append to reduce token usage during inference. Training with ThinkFree-adapted inputs improves performance and lowers token consumption even in original slow-thinking mode.

Result: TFPI accelerates RL convergence, achieves higher performance ceiling, yields more token-efficient reasoning models without specialized rewards or complex training designs. A 4B model trained with TFPI reaches 89.0% accuracy on AIME24 and 65.5% on LiveCodeBench using less than 4K H20 hours.

Conclusion: TFPI provides a simple yet effective adaptation to RLVR that bridges CoT distillation and standard RLVR, enabling more efficient training and inference while maintaining or improving performance.

Abstract: Reinforcement Learning with Verifiable Reward (RLVR) effectively solves complex tasks but demands extremely long context lengths during training, leading to substantial computational costs. While multi-stage training can partially mitigate this, starting with overly short contexts often causes irreversible performance degradation, ultimately failing to reduce overall training compute significantly. In this paper, we introduce Thinking-Free Policy Initialization (TFPI), a simple yet effective adaptation to RLVR that bridges long Chain-of-Thought (CoT) distillation and standard RLVR. TFPI employs a simple ThinkFree operation, explicitly discarding the thinking content via a direct append, to reduce token usage during inference. Training with ThinkFree-adapted inputs improves performance and lowers token consumption, even in the original slow-thinking mode. Extensive experiments across various benchmarks have shown that TFPI accelerates RL convergence, achieves a higher performance ceiling, and yields more token-efficient reasoning models without specialized rewards or complex training designs. With TFPI only, we train a 4B model to reach 89.0% accuracy on AIME24 and 65.5% on LiveCodeBench using less than 4K H20 hours.

Method: 1) Analyzed TCGA-LAML RNA-seq data using WGCNA to identify 20 high-value biomarkers; 2) Modeled target structures with AlphaFold3; 3) Identified druggable hotspots with DOGSiteScorer; 4) Developed reaction-first evolutionary metaheuristic algorithm with multi-objective optimization to assemble novel ligands from fragment libraries; 5) Validated candidates through ADMET profiling and molecular docking.

Result: Generated structurally unique chemical entities with drug-like properties (QED scores 0.5-0.7). Identified high-confidence candidates like Ligand L1 with binding free energy of -6.571 kcal/mol against A08A96 biomarker. Demonstrated pharmacologically viable, patient-tailored leads.

Conclusion: Integrating systems biology with metaheuristic molecular assembly can produce scalable, personalized drug leads for AML, offering a blueprint for precision oncology beyond AML.

Abstract: Acute Myeloid Leukemia (AML) remains a clinical challenge due to its extreme molecular heterogeneity and high relapse rates. While precision medicine has introduced mutation-specific therapies, many patients still lack effective, personalized options. This paper presents a novel, end-to-end computational framework that bridges the gap between patient-specific transcriptomics and de novo drug discovery. By analyzing bulk RNA sequencing data from the TCGA-LAML cohort, the study utilized Weighted Gene Co-expression Network Analysis (WGCNA) to prioritize 20 high-value biomarkers, including metabolic transporters like HK3 and immune-modulatory receptors such as SIGLEC9. The physical structures of these targets were modeled using AlphaFold3, and druggable hotspots were quantitatively mapped via the DOGSiteScorer engine. Then developed a novel, reaction-first evolutionary metaheuristic algorithm as well as multi-objective optimization programming that assembles novel ligands from fragment libraries, guided by spatial alignment to these identified hotspots. The generative model produced structurally unique chemical entities with a strong bias toward drug-like space, as evidenced by QED scores peaking between 0.5 and 0.7. Validation through ADMET profiling and SwissDock molecular docking identified high-confidence candidates, such as Ligand L1, which achieved a binding free energy of -6.571 kcal/mol against the A08A96 biomarker. These results demonstrate that integrating systems biology with metaheuristic molecular assembly can produce pharmacologically viable, patient tailored leads, offering a scalable blueprint for precision oncology in AML and beyond

Method: Learns a local update operator conditioned on neural shape attributes, trained once on a single representative shape, that integrates naturally with neural surface representations and generalizes across shape variations.

Result: Slightly outperforms CPM while remaining reasonably close to FEM accuracy, works across different neural representations, and provides the first end-to-end pipeline for solving surface PDEs on both neural and classical representations.

Conclusion: Enables accurate, fast inference of surface PDEs directly on neural representations without explicit meshing or per-instance optimization while preserving differentiability for end-to-end workflows.

Abstract: Solving partial differential equations (PDEs) on shapes underpins many shape analysis and engineering tasks; yet, prevailing PDE solvers operate on polygonal/triangle meshes while modern 3D assets increasingly live as neural representations. This mismatch leaves no suitable method to solve surface PDEs directly within the neural domain, forcing explicit mesh extraction or per-instance residual training, preventing end-to-end workflows. We present a novel, mesh-free formulation that learns a local update operator conditioned on neural (local) shape attributes, enabling surface PDEs to be solved directly where the (neural) data lives. The operator integrates naturally with prevalent neural surface representations, is trained once on a single representative shape, and generalizes across shape and topology variations, enabling accurate, fast inference without explicit meshing or per-instance optimization while preserving differentiability. Across analytic benchmarks (heat equation and Poisson solve on sphere) and real neural assets across different representations, our method slightly outperforms CPM while remaining reasonably close to FEM, and, to our knowledge, delivers the first end-to-end pipeline that solves surface PDEs on both neural and classical surface representations. Code will be released on acceptance.

Method: Theoretical study of binary classification with a classifier connected to optimal Bayes classifier, proving existence of beneficial pre-classification processing for finite training samples. Empirical investigation of theoretical setup and practical study with denoising/encoding on deep classifiers with varying training size, class distribution, and noise levels.

Result: Proved that for any finite number of training samples, there exists pre-classification processing that improves classification accuracy. Explored effects of class separation, training set size, and class balance on relative gain. Empirical studies showed consistent trends with theoretical results.

Conclusion: While data processing inequality holds for optimal Bayes classifiers, practical classifiers with finite training samples can benefit from preprocessing, explaining why low-level processing before high-level tasks remains common despite modern deep networks’ capabilities.

Abstract: The data processing inequality is an information-theoretic principle stating that the information content of a signal cannot be increased by processing the observations. In particular, it suggests that there is no benefit in enhancing the signal or encoding it before addressing a classification problem. This assertion can be proven to be true for the case of the optimal Bayes classifier. However, in practice, it is common to perform “low-level” tasks before “high-level” downstream tasks despite the overwhelming capabilities of modern deep neural networks. In this paper, we aim to understand when and why low-level processing can be beneficial for classification. We present a comprehensive theoretical study of a binary classification setup, where we consider a classifier that is tightly connected to the optimal Bayes classifier and converges to it as the number of training samples increases. We prove that for any finite number of training samples, there exists a pre-classification processing that improves the classification accuracy. We also explore the effect of class separation, training set size, and class balance on the relative gain from this procedure. We support our theory with an empirical investigation of the theoretical setup. Finally, we conduct an empirical study where we investigate the effect of denoising and encoding on the performance of practical deep classifiers on benchmark datasets. Specifically, we vary the size and class distribution of the training set, and the noise level, and demonstrate trends that are consistent with our theoretical results.

Method: A novel 3-phase block-level WSD training scheme: progressive increasing block-size diffusion (warm-up), large-scale full-sequence diffusion (stable), and reverting to compact-size block diffusion (decay). Post-training alignment with SFT and DPO to create instruction-tuned MoE variants.

Result: Created LLaDA2.0-mini (16B) and LLaDA2.0-flash (100B) instruction-tuned MoE models that preserve parallel decoding advantages and deliver superior performance and efficiency at frontier scale. Both models were open-sourced.

Conclusion: LLaDA2.0 establishes a new paradigm for converting auto-regressive models to discrete diffusion models, enabling efficient frontier-scale deployment with knowledge inheritance and superior performance through parallel decoding.

Abstract: This paper presents LLaDA2.0 – a tuple of discrete diffusion large language models (dLLM) scaling up to 100B total parameters through systematic conversion from auto-regressive (AR) models – establishing a new paradigm for frontier-scale deployment. Instead of costly training from scratch, LLaDA2.0 upholds knowledge inheritance, progressive adaption and efficiency-aware design principle, and seamless converts a pre-trained AR model into dLLM with a novel 3-phase block-level WSD based training scheme: progressive increasing block-size in block diffusion (warm-up), large-scale full-sequence diffusion (stable) and reverting back to compact-size block diffusion (decay). Along with post-training alignment with SFT and DPO, we obtain LLaDA2.0-mini (16B) and LLaDA2.0-flash (100B), two instruction-tuned Mixture-of-Experts (MoE) variants optimized for practical deployment. By preserving the advantages of parallel decoding, these models deliver superior performance and efficiency at the frontier scale. Both models were open-sourced.

Method: Proposes TimeBridge framework using diffusion bridges to learn paths between chosen priors and data distribution. Explores several prior designs: (i) data- and time-dependent priors for unconditional generation, and (ii) scale-preserving priors for conditional generation.

Result: Experiments show that TimeBridge with data-driven priors outperforms standard diffusion models on time series generation tasks.

Conclusion: TimeBridge provides a flexible framework for time series generation with tailored priors that better capture time series properties, offering improved performance over standard diffusion approaches.

Abstract: Time series generation is widely used in real-world applications such as simulation, data augmentation, and hypothesis testing. Recently, diffusion models have emerged as the de facto approach to time series generation, enabling diverse synthesis scenarios. However, the fixed standard-Gaussian diffusion prior may be ill-suited for time series data, which exhibit properties such as temporal order and fixed time points. In this paper, we propose TimeBridge, a framework that flexibly synthesizes time series data by using diffusion bridges to learn paths between a chosen prior and the data distribution. We then explore several prior designs tailored to time series synthesis. Our framework covers (i) data- and time-dependent priors for unconditional generation and (ii) scale-preserving priors for conditional generation. Experiments show that our framework with data-driven priors outperforms standard diffusion models on time series generation.

Method: Formalizes the problem as reinforcement learning, introduces a scalable benchmark, encodes images as overcomplete sparse codes, and provides theoretical justification for the approach.

Result: Reinforcement learning with sparse coding representations can efficiently solve optimal control tasks orders of magnitude larger than those solvable with complete codes.

Conclusion: Deep learning is not necessary for efficient optimal control with natural images; sparse coding representations combined with reinforcement learning provide an effective alternative.

Abstract: Optimal control and sequential decision making are widely used in many complex tasks. Optimal control over a sequence of natural images is a first step towards understanding the role of vision in control. Here, we formalize this problem as a reinforcement learning task, and derive general conditions under which an image includes enough information to implement an optimal policy. Reinforcement learning is shown to provide a computationally efficient method for finding optimal policies when natural images are encoded into “efficient” image representations. This is demonstrated by introducing a new reinforcement learning benchmark that easily scales to large numbers of states and long horizons. In particular, by representing each image as an overcomplete sparse code, we are able to efficiently solve an optimal control task that is orders of magnitude larger than those tasks solvable using complete codes. Theoretical justification for this behaviour is provided. This work also demonstrates that deep learning is not necessary for efficient optimal control with natural images.

Method: Uses Growing-Window estimator to sample from environment interactions to estimate stochastic thresholds, then applies Stochastic Pessimistic-Optimistic Thresholding (SPOT) - a model-based primal-dual algorithm for multiple constraints against stochastic thresholds.

Result: Achieves sublinear regret and constraint violation: $\tilde{\mathcal{O}}(\sqrt{T})$ reward regret and $\tilde{\mathcal{O}}(\sqrt{T})$ constraint violation over T episodes, comparable to approaches with fixed thresholds.

Conclusion: SPOT is the first RL algorithm with theoretical guarantees for uncertain environments where thresholds are unknown, enabling safe learning under both pessimistic and optimistic threshold settings.

Abstract: This paper studies constrained Markov decision processes (CMDPs) with constraints against stochastic thresholds, aiming at safety of reinforcement learning in unknown and uncertain environments. We leverage a Growing-Window estimator sampling from interactions with the uncertain environment to estimate the thresholds, based on which we design Stochastic Pessimistic-Optimistic Thresholding (SPOT), a novel model-based primal-dual algorithm for multiple constraints against stochastic thresholds. SPOT enables reinforcement learning under both pessimistic and optimistic threshold settings. We prove that our algorithm achieves sublinear regret and constraint violation; i.e., a reward regret of $\tilde{\mathcal{O}}(\sqrt{T})$ while allowing an $\tilde{\mathcal{O}}(\sqrt{T})$ constraint violation over $T$ episodes. The theoretical guarantees show that our algorithm achieves performance comparable to that of an approach relying on fixed and clear thresholds. To the best of our knowledge, SPOT is the first reinforcement learning algorithm that realises theoretical guaranteed performance in an uncertain environment where even thresholds are unknown.

Method: Proposed framework for weighted aggregation of prediction sets where weights are assigned based on each set’s contribution. Derives procedure for weighted aggregation that maintains finite-sample validity even with data-dependent weights, making it applicable to learned-weight settings like mixture-of-experts.

Result: Framework achieves tighter coverage bounds that interpolate between 1-2α (combined models) and 1-α (individual model) depending on weight distribution. Maintains finite-sample validity with data-dependent weights and demonstrates adaptive coverage in mixture-of-experts experiments.

Conclusion: Weighted aggregation framework provides flexible control over prediction set aggregation with improved coverage guarantees, generalizes to data-dependent weights, and enables adaptive coverage in practical applications like mixture-of-experts models.

Abstract: Conformal prediction quantifies the uncertainty of machine learning models by augmenting point predictions with valid prediction sets. For complex scenarios involving multiple trials, models, or data sources, conformal prediction sets can be aggregated to create a prediction set that captures the overall uncertainty, often improving precision. However, aggregating multiple prediction sets with individual $1-α$ coverage inevitably weakens the overall guarantee, typically resulting in $1-2α$ worst-case coverage. In this work, we propose a framework for the weighted aggregation of prediction sets, where weights are assigned to each prediction set based on their contribution. Our framework offers flexible control over how the sets are aggregated, achieving tighter coverage bounds that interpolate between the $1-2α$ guarantee of the combined models and the $1-α$ guarantee of an individual model depending on the distribution of weights. Importantly, our framework generalizes to data-dependent weights, as we derive a procedure for weighted aggregation that maintains finite-sample validity even when the weights depend on the data. This extension makes our framework broadly applicable to settings where weights are learned, such as mixture-of-experts (MoE), and we demonstrate through experiments in the MoE setting that our methods achieve adaptive coverage.

Method: Represent AI models as DAGs, reformulate splitting as minimum s-t cut problem, propose fast DAG-based algorithm using maximum flow method, and block-wise algorithm for structured models.

Result: Algorithms find optimal splitting in milliseconds, reduce training delay by 24.62%-38.95% vs benchmarks in dynamic edge networks.

Conclusion: Proposed DAG-based approach provides efficient optimal model splitting for split learning with significant performance improvements.

Abstract: Split learning (SL) has emerged as a computationally efficient approach for artificial intelligence (AI) model training, which can alleviate device-side computational workloads. However, complex AI model architectures pose high computational complexity to obtain the optimal model splitting. In this paper, we represent an arbitrary AI model as a directed acyclic graph (DAG), and then reformulate the optimal model splitting problem as a minimum s-t cut search problem. To solve the problem, we propose a fast DAG-based model splitting algorithm, which restructures the DAG to enable the optimal model splitting identification via a maximum flow method. Theoretical analysis indicates that the proposed algorithm is optimal. Furthermore, considering AI models with block structures, we propose a block-wise model splitting algorithm to reduce computational complexity. The algorithm abstracts each block, i.e., a component consisting of multiple layers, into a single vertex, thereby obtaining the optimal model splitting via a simplified DAG. Extensive experimental results demonstrate that the proposed algorithms can determine the optimal model splitting within milliseconds, as well as reduce training delay by 24.62%-38.95% in dynamic edge networks as compared to the state-of-the-art benchmarks.

Method: Random Bernoulli selection between SILU and RELU activations during training, with RELU-only inference for sparsity; also applied to sequence generation for diversity.

Result: Better than training from scratch with RELU alone; significant CPU/GPU speedup from sparse inference; higher diversity in text generation with only slight performance drop vs. SILU.

Conclusion: Stochastic activations solve RELU optimization issues, enable efficient sparse inference, and provide alternative text diversity method with minimal performance trade-off.

Abstract: We introduce stochastic activations. This novel strategy randomly selects between several non-linear functions in the feed-forward layer of a large language model. In particular, we choose between SILU or RELU depending on a Bernoulli draw. This strategy circumvents the optimization problem associated with RELU, namely, the constant shape for negative inputs that prevents the gradient flow. We leverage this strategy in two ways: (1) We use stochastic activations during pre-training and fine-tune the model with RELU, which is used at inference time to provide sparse latent vectors. This reduces the inference FLOPs and translates into a significant speedup on CPU and GPU. This leads to better results than training from scratch with the RELU activation function. (2) We evaluate stochastic activations for sequence generation. This strategy performs reasonably well: it has higher diversity and has only slightly inferior performance to the best deterministic non-linearity, SILU, combined with temperature sampling. This provides an alternative way to increase the diversity of generated text.

Method: Designs a sparse projection method that sets sparsity level explicitly for whole vector sets, projects groups simultaneously with automatic per-vector sparsity tuning, and generalizes to weighted ℓ₁ norms.

Result: Method has linear computational complexity. In ResNet50 pruning, produces sparse models with significantly higher accuracy at corresponding sparsity levels. In nonnegative matrix factorization, yields competitive reconstruction errors.

Conclusion: The proposed sparse projection method effectively controls average sparsity for vector sets, outperforms existing approaches in deep learning pruning, and shows competitive performance in matrix factorization tasks.

Abstract: We design a new sparse projection method for a set of vectors that guarantees a desired average sparsity level measured leveraging the popular Hoyer measure (an affine function of the ratio of the $\ell_1$ and $\ell_2$ norms). Existing approaches either project each vector individually or require the use of a regularization parameter which implicitly maps to the average $\ell_0$-measure of sparsity. Instead, in our approach we set the sparsity level for the whole set explicitly and simultaneously project a group of vectors with the sparsity level of each vector tuned automatically. We show that the computational complexity of our projection operator is linear in the size of the problem. Additionally, we propose a generalization of this projection by replacing the $\ell_1$ norm by its weighted version. We showcase the efficacy of our approach in both supervised and unsupervised learning tasks on image datasets including CIFAR10 and ImageNet. In deep neural network pruning, the sparse models produced by our method on ResNet50 have significantly higher accuracies at corresponding sparsity values compared to existing competitors. In nonnegative matrix factorization, our approach yields competitive reconstruction errors against state-of-the-art algorithms.

Method: Formalize verifier unreliability as stochastic reward channel with asymmetric noise rates (FP rate ρ₀, FN rate ρ₁). Propose two lightweight corrections: 1) backward correction yields unbiased surrogate reward for unbiased policy-gradient estimator, 2) forward correction reweights score-function terms to align with clean gradient direction (requires only FN rate). Implement both as hooks in group relative policy optimization pipeline.

Result: Both corrections improve RLVR for math reasoning under synthetic and real verifier noise. Forward variant is more stable under heavier noise. Appeals mechanism with lightweight LLM verifier estimates FN rate online and further improves performance.

Conclusion: The proposed lightweight corrections effectively address verifier unreliability in RLVR systems, with forward correction offering better stability under heavy noise. Online FN rate estimation via appeals mechanism provides additional performance improvements.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) replaces costly human labeling with automated verifiers. To reduce verifier hacking, many RLVR systems binarize rewards to ${0,1}$, but imperfect verifiers inevitably introduce \emph{false negatives} (rejecting correct answers) and \emph{false positives} (accepting incorrect ones). We formalize verifier unreliability as a stochastic reward channel with asymmetric noise rates $ρ_0$ and $ρ_1$ – the FP rate and the FN rate, respectively. From this abstraction we derive two lightweight corrections: (i) a \emph{backward} correction that yields an unbiased surrogate reward and thus an unbiased policy-gradient estimator in expectation, and (ii) a \emph{forward} correction that reweights score-function terms so the expected update aligns with the clean gradient direction and requires only the FN rate. We implement both as lightweight hooks in a group relative policy optimization pipeline, both corrections improve RLVR for math reasoning under synthetic and real verifier noise, with the forward variant being more stable under heavier noise. Finally, an appeals mechanism with a lightweight LLM verifier estimates the FN rate online and further improves performance.

Method: DATTA uses three components: 1) domain-diversity discriminator to recognize single/multiple-domain patterns, 2) domain-diversity adaptive batch normalization combining source and test-time statistics, and 3) domain-diversity adaptive fine-tuning to resolve gradient conflicts.

Result: Extensive experiments show DATTA significantly outperforms state-of-the-art methods by up to 13% in handling dynamic domain shift data streams.

Conclusion: DATTA is the first approach to successfully handle TTA under dynamic domain shift data streams, providing a robust solution for real-world applications where domain distributions change over time.

Abstract: Test-Time Adaptation (TTA) addresses domain shifts between training and testing. However, existing methods assume a homogeneous target domain (e.g., single domain) at any given time. They fail to handle the dynamic nature of real-world data, where single-domain and multiple-domain distributions change over time. We identify that performance drops in multiple-domain scenarios are caused by batch normalization errors and gradient conflicts, which hinder adaptation. To solve these challenges, we propose Domain Diversity Adaptive Test-Time Adaptation (DATTA), the first approach to handle TTA under dynamic domain shift data streams. It is guided by a novel domain-diversity score. DATTA has three key components: a domain-diversity discriminator to recognize single- and multiple-domain patterns, domain-diversity adaptive batch normalization to combine source and test-time statistics, and domain-diversity adaptive fine-tuning to resolve gradient conflicts. Extensive experiments show that DATTA significantly outperforms state-of-the-art methods by up to 13%. Code is available at https://github.com/DYW77/DATTA.

Method: Proposes H-embedding (transferability-aware task embedding) derived from information theoretic measure of transferability. Constructs hypernet framework under H-embedding guidance to learn task-conditioned model weights. H-embedding is online and easy to compute.

Result: Extensive evaluations on CIFAR-100, ImageNet-R, and DomainNet show prominent performance compared to various baseline and SOTA approaches. Demonstrates strong potential in capturing and utilizing intrinsic task relationships.

Conclusion: The proposed H-embedding and hypernet framework effectively enhance continual learning by leveraging inter-task relationships through transferability-aware embeddings, showing practical advantages with low-dimensional storage and efficient end-to-end training.

Abstract: Continual learning (CL) has been a critical topic in contemporary deep neural network applications, where higher levels of both forward and backward transfer are desirable for an effective CL performance. Existing CL strategies primarily focus on task models, either by regularizing model updates or by separating task-specific and shared components, while often overlooking the potential of leveraging inter-task relationships to enhance transfer. To address this gap, we propose a transferability-aware task embedding, termed H-embedding, and construct a hypernet framework under its guidance to learn task-conditioned model weights for CL tasks. Specifically, H-embedding is derived from an information theoretic measure of transferability and is designed to be online and easy to compute. Our method is also characterized by notable practicality, requiring only the storage of a low-dimensional task embedding per task and supporting efficient end-to-end training. Extensive evaluations on benchmarks including CIFAR-100, ImageNet-R, and DomainNet show that our framework performs prominently compared to various baseline and SOTA approaches, demonstrating strong potential in capturing and utilizing intrinsic task relationships. Our code is publicly available at https://github.com/viki760/H-embedding-Guided-Hypernet.

Method: PEARL dynamically allocates ranks for LoRA components during continual learning training. It uses reference task weights and adaptively determines rank based on current task’s proximity to reference weights in parameter space.

Result: PEARL outperforms all considered baselines by a large margin across three vision architectures (ResNet, Separable Convolutional Network, Vision Transformer) and multiple continual learning scenarios.

Conclusion: PEARL provides an effective rehearsal-free continual learning solution with dynamic rank allocation that addresses LoRA’s rank sensitivity problem while maintaining parameter efficiency and preventing catastrophic forgetting.

Abstract: Catastrophic forgetting has remained a critical challenge for deep neural networks in Continual Learning (CL) as it undermines consolidated knowledge when learning new tasks. Parameter efficient fine tuning CL techniques are gaining traction for their effectiveness in addressing catastrophic forgetting with a lightweight training schedule while avoiding degradation of consolidated knowledge in pre-trained models. However, low rank adapters (LoRA) in these approaches are highly sensitive to rank selection which can lead to sub-optimal resource allocation and performance. To this end, we introduce PEARL, a rehearsal-free CL framework that entails dynamic rank allocation for LoRA components during CL training. Specifically, PEARL leverages reference task weights and adaptively determines the rank of task-specific LoRA components based on the current tasks’ proximity to reference task weights in parameter space. To demonstrate the versatility of PEARL, we evaluate it across three vision architectures (ResNet, Separable Convolutional Network and Vision Transformer) and a multitude of CL scenarios, and show that PEARL outperforms all considered baselines by a large margin.

Method: Proposes an automated approach for path loss model formulation, evaluation, and refinement using two AI techniques: 1) Deep Symbolic Regression for fully interpretable models, and 2) Kolmogorov-Arnold Networks offering two levels of interpretability. Both methods are evaluated on two synthetic and two real-world datasets.

Result: Kolmogorov-Arnold Networks achieve R^2 close to 1 with minimal prediction error, while Deep Symbolic Regression generates compact models with moderate accuracy. Automated methods outperform traditional approaches, achieving up to 75% reduction in prediction errors.

Conclusion: The proposed AI-based automated methods provide accurate and explainable solutions for path loss modeling, potentially increasing efficiency in discovering next-generation propagation models for 5G+ wireless systems.

Abstract: Modeling propagation is the cornerstone for designing and optimizing next-generation wireless systems, with a particular emphasis on 5G and beyond era. Traditional modeling methods have long relied on statistic-based techniques to characterize propagation behavior across different environments. With the expansion of wireless communication systems, there is a growing demand for methods that guarantee the accuracy and interpretability of modeling. Artificial intelligence (AI)-based techniques, in particular, are increasingly being adopted to overcome this challenge, although the interpretability is not assured with most of these methods. Inspired by recent advancements in AI, this paper proposes a novel approach that accelerates the discovery of path loss models while maintaining interpretability. The proposed method automates the formulation, evaluation, and refinement of the model, facilitating the discovery of the model. We examine two techniques: one based on Deep Symbolic Regression, offering full interpretability, and the second based on Kolmogorov-Arnold Networks, providing two levels of interpretability. Both approaches are evaluated on two synthetic and two real-world datasets. Our results show that Kolmogorov-Arnold Networks achieve the coefficient of determination value R^2 close to 1 with minimal prediction error, while Deep Symbolic Regression generates compact models with moderate accuracy. Moreover, on the selected examples, we demonstrate that automated methods outperform traditional methods, achieving up to 75% reduction in prediction errors, offering accurate and explainable solutions with potential to increase the efficiency of discovering next-generation path loss models.

Method: Alternating Gradient Flows (AGF) framework models training dynamics as a two-step alternating process: (1) dormant neurons activate by maximizing a utility function, triggering feature acquisition and loss drops; (2) active neurons optimize by minimizing a cost function. This approximates the observed staircase loss patterns.

Result: AGF successfully quantifies the order, timing, and magnitude of loss drops, matching experimental observations across various architectures. It unifies existing saddle-to-saddle analyses, proves convergence to gradient flow in diagonal linear networks, and provides the first complete characterization of training dynamics in quadratic networks performing modular addition, revealing Fourier feature learning by coefficient magnitude.

Conclusion: AGF offers a promising framework for understanding feature learning in neural networks by explaining staircase loss patterns, feature acquisition order, and providing theoretical unification across different network architectures.

Abstract: What features neural networks learn, and how, remains an open question. In this paper, we introduce Alternating Gradient Flows (AGF), an algorithmic framework that describes the dynamics of feature learning in two-layer networks trained from small initialization. Prior works have shown that gradient flow in this regime exhibits a staircase-like loss curve, alternating between plateaus where neurons slowly align to useful directions and sharp drops where neurons rapidly grow in norm. AGF approximates this behavior as an alternating two-step process: maximizing a utility function over dormant neurons and minimizing a cost function over active ones. AGF begins with all neurons dormant. At each iteration, a dormant neuron activates, triggering the acquisition of a feature and a drop in the loss. AGF quantifies the order, timing, and magnitude of these drops, matching experiments across several commonly studied architectures. We show that AGF unifies and extends existing saddle-to-saddle analyses in fully connected linear networks and attention-only linear transformers, where the learned features are singular modes and principal components, respectively. In diagonal linear networks, we prove AGF converges to gradient flow in the limit of vanishing initialization. Applying AGF to quadratic networks trained to perform modular addition, we give the first complete characterization of the training dynamics, revealing that networks learn Fourier features in decreasing order of coefficient magnitude. Altogether, AGF offers a promising step towards understanding feature learning in neural networks.

Method: DaSH (Dataset Selection via Hierarchies) models utility hierarchically at both dataset and group levels (e.g., collections, institutions). This enables efficient generalization from limited observations by leveraging the hierarchical structure of data sources.

Result: DaSH outperforms state-of-the-art data selection baselines by up to 26.2% accuracy on Digit-Five and DomainNet benchmarks. It requires significantly fewer exploration steps and shows robustness in low-resource settings and when relevant datasets are scarce.

Conclusion: DaSH provides an effective solution for scalable and adaptive dataset selection in practical multi-source learning workflows, addressing the critical need to select entire datasets rather than individual samples from heterogeneous data pools.

Abstract: The success of modern machine learning hinges on access to high-quality training data. In many real-world scenarios, such as acquiring data from public repositories or sharing across institutions, data is naturally organized into discrete datasets that vary in relevance, quality, and utility. Selecting which repositories or institutions to search for useful datasets, and which datasets to incorporate into model training are therefore critical decisions, yet most existing methods select individual samples and treat all data as equally relevant, ignoring differences between datasets and their sources. In this work, we formalize the task of dataset selection: selecting entire datasets from a large, heterogeneous pool to improve downstream performance under resource constraints. We propose Dataset Selection via Hierarchies (DaSH), a dataset selection method that models utility at both dataset and group (e.g., collections, institutions) levels, enabling efficient generalization from limited observations. Across two public benchmarks (Digit-Five and DomainNet), DaSH outperforms state-of-the-art data selection baselines by up to 26.2% in accuracy, while requiring significantly fewer exploration steps. Ablations show DaSH is robust to low-resource settings and lack of relevant datasets, making it suitable for scalable and adaptive dataset selection in practical multi-source learning workflows.

Method: Combines physics-informed neural networks with hyper-networks (to construct ODE solver) and Wasserstein generative adversarial networks (to estimate ODE parameters by capturing noisy data distributions).

Result: SiGMoID successfully quantifies data noise, estimates system parameters, and infers unobserved system components, validated through realistic experimental examples.

Conclusion: The approach enables precise and robust inference for dynamic systems with broad applicability across scientific research and engineered systems, facilitating discovery of full system dynamics.

Abstract: System inference for nonlinear dynamic models, represented by ordinary differential equations (ODEs), remains a significant challenge in many fields, particularly when the data are noisy, sparse, or partially observable. In this paper, we propose a Simulation-based Generative Model for Imperfect Data (SiGMoID) that enables precise and robust inference for dynamic systems. The proposed approach integrates two key methods: (1) physics-informed neural networks with hyper-networks that constructs an ODE solver, and (2) Wasserstein generative adversarial networks that estimates ODE parameters by effectively capturing noisy data distributions. We demonstrate that SiGMoID quantifies data noise, estimates system parameters, and infers unobserved system components. Its effectiveness is validated validated through realistic experimental examples, showcasing its broad applicability in various domains, from scientific research to engineered systems, and enabling the discovery of full system dynamics.

Method: AdaMuon combines two tightly coupled mechanisms: (1) element-wise second momentum estimator applied to orthogonalized update directions, and (2) sign-stabilized orthogonal update where momentum is sign-transformed before orthogonalization. It also uses RMS-aligned rescaling to match Adam’s update magnitude for compatibility with existing learning rate schedules.

Result: Experiments show AdaMuon maintains stability and surpasses Adam by more than 40% training efficiency in large-scale scenarios, demonstrating superior performance without requiring extra hyperparameter tuning.

Conclusion: AdaMuon successfully combines element-wise adaptivity with orthogonal updates to achieve both stability and significant efficiency gains over Adam in large-scale neural network training, while maintaining compatibility with existing learning rate schedules.

Abstract: We propose AdaMuon, a novel optimizer that combines element-wise adaptivity with orthogonal updates for large-scale neural network training. AdaMuon incorporates two tightly coupled mechanisms: (1) an element-wise second momentum estimator applied to orthogonalized update directions, and (2) a sign-stabilized orthogonal update, where the momentum is first sign-transformed before orthogonalization. These two components jointly enable variance-adaptive scaling while maintaining stable update geometry. In addition, AdaMuon employs an RMS-aligned rescaling strategy to match the root-mean-square update magnitude to Adam, allowing direct reuse of existing learning rate schedules without extra tuning. Experiments demonstrate that AdaMuon not only maintains stability but can surpass Adam by more than 40% training efficiency in large-scale scenarios.

Method: Benchmarked Expected Hypervolume Improvement (EHVI) against fixed-weight scalarized Expected Improvement (EI) under controlled setup with identical Gaussian Process surrogates and molecular representations across three molecular optimization tasks.

Result: EHVI consistently outperformed scalarized EI in terms of Pareto front coverage, convergence speed, and chemical diversity. Even strong deterministic scalarization variants underperform in low-data regimes.

Conclusion: Pareto-aware acquisition (EHVI) offers practical advantages for de novo molecular optimization, especially with limited evaluation budgets and nontrivial trade-offs.

Abstract: Multi-objective Bayesian optimization (MOBO) provides a principled framework for navigating trade-offs in molecular design. However, its empirical advantages over scalarized alternatives remain underexplored. We benchmark a simple Pareto-based MOBO strategy - Expected Hypervolume Improvement (EHVI) - against a simple fixed-weight scalarized baseline using Expected Improvement (EI), under a tightly controlled setup with identical Gaussian Process surrogates and molecular representations. Across three molecular optimization tasks, EHVI consistently outperforms scalarized EI in terms of Pareto front coverage, convergence speed, and chemical diversity. While scalarization encompasses flexible variants - including random or adaptive schemes - our results show that even strong deterministic instantiations can underperform in low-data regimes. These findings offer concrete evidence for the practical advantages of Pareto-aware acquisition in de novo molecular optimization, especially when evaluation budgets are limited and trade-offs are nontrivial.

Method: Evaluated LASSO logistic regression, random forest, XGBoost, and neural networks using clinical feature subsets (including top 10 SHAP-ranked features). Applied equal opportunity postprocessing to reduce disparities in true positive rates across racial/ethnic subgroups.

Result: Selected LASSO logistic regression with top 10 features for interpretability and comparable performance. Before fairness adjustment: AUROC 0.84, accuracy 78%, sensitivity 72%, specificity 79%, F1-score 0.617. After adjustment: accuracy 81%, specificity 94%, sensitivity 41%, F1-score 0.515.

Conclusion: MASER model achieved competitive performance (AUROC 0.836, accuracy 77.6%) comparable to ensemble/tree-based models. Demonstrates interpretable models can balance predictive performance and fairness in diverse populations.

Abstract: Background: Metabolic Dysfunction-Associated Steatotic Liver Disease (MASLD) affects ~33% of U.S. adults and is the most common chronic liver disease. Although often asymptomatic, progression can lead to cirrhosis. Early detection is important, as lifestyle interventions can prevent disease progression. We developed a fair, rigorous, and reproducible MASLD prediction model and compared it to prior methods using a large electronic health record database. Methods: We evaluated LASSO logistic regression, random forest, XGBoost, and a neural network for MASLD prediction using clinical feature subsets, including the top 10 SHAP-ranked features. To reduce disparities in true positive rates across racial and ethnic subgroups, we applied an equal opportunity postprocessing method. Results: This study included 59,492 patients in the training data, 24,198 in the validating data, and 25,188 in the testing data. The LASSO logistic regression model with the top 10 features was selected for its interpretability and comparable performance. Before fairness adjustment, the model achieved AUROC of 0.84, accuracy of 78%, sensitivity of 72%, specificity of 79%, and F1-score of 0.617. After equal opportunity postprocessing, accuracy modestly increased to 81% and specificity to 94%, while sensitivity decreased to 41% and F1-score to 0.515, reflecting the fairness trade-off. Conclusions: We developed the MASER prediction model (MASLD Static EHR Risk Prediction), a LASSO logistic regression model which achieved competitive performance for MASLD prediction (AUROC 0.836, accuracy 77.6%), comparable to previously reported ensemble and tree-based models. Overall, this approach demonstrates that interpretable models can achieve a balance of predictive performance and fairness in diverse patient populations.

Method: Introduces a higher-order, non-causal sequence model that outputs controllers which manipulate the residual stream activations of a base autoregressive model. These controllers execute temporally-abstract actions with learned termination conditions.

Result: The higher-order model learns to compress long activation sequences into internal controllers that execute behaviorally meaningful action sequences over long timescales. Internal RL enables learning from sparse rewards where standard RL fails, demonstrated on grid world and MuJoCo tasks.

Conclusion: Internal RL is a promising approach for hierarchical RL within foundation models, enabling efficient exploration and learning from sparse rewards through latent action generation and reinforcement in internal representations.

Abstract: Large-scale autoregressive models pretrained on next-token prediction and finetuned with reinforcement learning (RL) have achieved unprecedented success on many problem domains. During RL, these models explore by generating new outputs, one token at a time. However, sampling actions token-by-token can result in highly inefficient learning, particularly when rewards are sparse. Here, we show that it is possible to overcome this problem by acting and exploring within the internal representations of an autoregressive model. Specifically, to discover temporally-abstract actions, we introduce a higher-order, non-causal sequence model whose outputs control the residual stream activations of a base autoregressive model. On grid world and MuJoCo-based tasks with hierarchical structure, we find that the higher-order model learns to compress long activation sequence chunks onto internal controllers. Critically, each controller executes a sequence of behaviorally meaningful actions that unfold over long timescales and are accompanied with a learned termination condition, such that composing multiple controllers over time leads to efficient exploration on novel tasks. We show that direct internal controller reinforcement, a process we term “internal RL”, enables learning from sparse rewards in cases where standard RL finetuning fails. Our results demonstrate the benefits of latent action generation and reinforcement in autoregressive models, suggesting internal RL as a promising avenue for realizing hierarchical RL within foundation models.

Method: Proposes Physics Informed Neural Networks (PINNs) with physics-based constraints, introduces novel Physics Informed Kolmogorov Arnold Network (PIKAN) blending universal function approximation theory with physical insights, uses structural parameters collected manually or via computer vision, and employs dataset of 15 real bridges augmented to 100 samples.

Result: Best model achieves R² score of 0.9603 and mean absolute error (MAE) of 10.50 units, demonstrating reliable weight estimation with limited data. Also provides web-based interface for parameter entry and prediction.

Conclusion: PINNs can offer reliable structural weight estimates even with limited data, potentially informing early-stage failure analysis in lightweight bridge designs. The approach shows promise for simplified structural models.

Abstract: Physics Informed Neural Networks (PINNs) are gaining attention for their ability to embed physical laws into deep learning models, which is particularly useful in structural engineering tasks with limited data. This paper aims to explore the use of PINNs to predict the weight of small scale spaghetti bridges, a task relevant to understanding load limits and potential failure modes in simplified structural models. Our proposed framework incorporates physics-based constraints to the prediction model for improved performance. In addition to standard PINNs, we introduce a novel architecture named Physics Informed Kolmogorov Arnold Network (PIKAN), which blends universal function approximation theory with physical insights. The structural parameters provided as input to the model are collected either manually or through computer vision methods. Our dataset includes 15 real bridges, augmented to 100 samples, and our best model achieves an $R^2$ score of 0.9603 and a mean absolute error (MAE) of 10.50 units. From applied perspective, we also provide a web based interface for parameter entry and prediction. These results show that PINNs can offer reliable estimates of structural weight, even with limited data, and may help inform early stage failure analysis in lightweight bridge designs. The complete data and code are available at https://github.com/OmerJauhar/PINNS-For-Spaghetti-Bridges.

Method: Introduces SynQuE problem formalization, establishes benchmarks with proxy metrics (distribution/diversity-based distance measures via embeddings), and proposes LENS proxy leveraging LLM reasoning for complex tasks.

Result: SynQuE proxies correlate with real task performance across diverse tasks; LENS consistently outperforms others on complex tasks; top-3 synthetic datasets selected via SynQuE raise Text2SQL accuracy from 30.4% to 38.4% (+8.1%).

Conclusion: Establishes SynQuE as practical framework for synthetic data selection under real-data scarcity, motivating future research on foundation model-based data characterization and fine-grained data selection.

Abstract: We introduce and formalize the Synthetic Dataset Quality Estimation (SynQuE) problem: ranking synthetic datasets by their expected real-world task performance using only limited unannotated real data. This addresses a critical and open challenge where data is scarce due to collection costs or privacy constraints. We establish the first comprehensive benchmarks for this problem by introducing and evaluating proxy metrics that choose synthetic data for training to maximize task performance on real data. We introduce the first proxy metrics for SynQuE by adapting distribution and diversity-based distance measures to our context via embedding models. To address the shortcomings of these metrics on complex planning tasks, we propose LENS, a novel proxy that leverages large language model reasoning. Our results show that SynQuE proxies correlate with real task performance across diverse tasks, including sentiment analysis, Text2SQL, web navigation, and image classification, with LENS consistently outperforming others on complex tasks by capturing nuanced characteristics. For instance, on text-to-SQL parsing, training on the top-3 synthetic datasets selected via SynQuE proxies can raise accuracy from 30.4% to 38.4 (+8.1)% on average compared to selecting data indiscriminately. This work establishes SynQuE as a practical framework for synthetic data selection under real-data scarcity and motivates future research on foundation model-based data characterization and fine-grained data selection.

Method: Integrates Kolmogorov-Arnold Networks (KANs) within a fair adversarial learning framework, leveraging KANs’ adversarial robustness and interpretability. Uses spline-based KAN architecture for stable adversarial optimization and proposes an adaptive fairness penalty update mechanism to balance fairness and accuracy.

Result: Empirical evidence on two real-world admissions datasets demonstrates the framework’s efficiency in achieving fairness across sensitive attributes while preserving predictive performance, with theoretical insights ensuring stability during adversarial optimization.

Conclusion: The proposed KAN-based fair adversarial learning framework effectively addresses fairness-accuracy trade-off challenges while providing interpretability, making it suitable for socially sensitive decision-making domains like college admissions.

Abstract: Despite recent advances in fairness-aware machine learning, predictive models often exhibit discriminatory behavior towards marginalized groups. Such unfairness might arise from biased training data, model design, or representational disparities across groups, posing significant challenges in high-stakes decision-making domains such as college admissions. While existing fair learning models aim to mitigate bias, achieving an optimal trade-off between fairness and accuracy remains a challenge. Moreover, the reliance on black-box models hinders interpretability, limiting their applicability in socially sensitive domains. To circumvent these issues, we propose integrating Kolmogorov-Arnold Networks (KANs) within a fair adversarial learning framework. Leveraging the adversarial robustness and interpretability of KANs, our approach facilitates stable adversarial learning. We derive theoretical insights into the spline-based KAN architecture that ensure stability during adversarial optimization. Additionally, an adaptive fairness penalty update mechanism is proposed to strike a balance between fairness and accuracy. We back these findings with empirical evidence on two real-world admissions datasets, demonstrating the proposed framework’s efficiency in achieving fairness across sensitive attributes while preserving predictive performance.

Method: Proposes a common design framework for representative shortcut models that provides theoretical justification for their validity while disentangling concrete component-level choices. This framework enables systematic identification of improvements without requiring pre-training, distillation, or curriculum learning.

Result: Achieves new state-of-the-art FID50k of 2.85 on ImageNet-256x256 with one-step generation under classifier-free guidance, and further reaches FID50k of 2.53 with 2x training steps. The model requires no pre-training, distillation, or curriculum learning.

Conclusion: The proposed framework lowers the barrier to component-level innovation in shortcut models and facilitates principled exploration of their design space, advancing one-step diffusion model performance while providing a systematic approach for future improvements.

Abstract: Recent advances in few-step diffusion models have demonstrated their efficiency and effectiveness by shortcutting the probabilistic paths of diffusion models, especially in training one-step diffusion models from scratch (\emph{a.k.a.} shortcut models). However, their theoretical derivation and practical implementation are often closely coupled, which obscures the design space. To address this, we propose a common design framework for representative shortcut models. This framework provides theoretical justification for their validity and disentangles concrete component-level choices, thereby enabling systematic identification of improvements. With our proposed improvements, the resulting one-step model achieves a new state-of-the-art FID50k of 2.85 on ImageNet-256x256 under the classifier-free guidance setting with one step generation, and further reaches FID50k of 2.53 with 2x training steps. Remarkably, the model requires no pre-training, distillation, or curriculum learning. We believe our work lowers the barrier to component-level innovation in shortcut models and facilitates principled exploration of their design space.

Method: Adapt arguments for complex VAEs with complex latent stage; derive Fisher information metric for complex Gaussian with trivial relation matrix; propose Kähler potential derivative of complex Gaussian mixtures as efficient proxy to Fisher metric; use law of total covariance to bridge potential and Fisher metric; regularize latent space with decoder geometry and sample with weighted complex volume element.

Result: Demonstrates that complex VAEs reveal Kähler geometric structure; enables efficient computation of Fisher information metric via Kähler potential derivatives; yields consistently smoother representations and fewer semantic outliers, though at the exchange of sample variation.

Conclusion: Complex VAEs naturally exhibit Kähler geometry, and leveraging this structure through efficient Kähler potential derivatives enables effective latent space regularization and improved representation quality while reducing computational burden.

Abstract: It has been discovered that latent-Euclidean variational autoencoders (VAEs) admit, in various capacities, Riemannian structure. We adapt these arguments but for complex VAEs with a complex latent stage. We show that complex VAEs reveal to some level Kähler geometric structure. Our methods will be tailored for decoder geometry. We derive the Fisher information metric in the complex case under a latent complex Gaussian with trivial relation matrix. It is well known from statistical information theory that the Fisher information coincides with the Hessian of the Kullback-Leibler (KL) divergence. Thus, the metric Kähler potential relation is exactly achieved under relative entropy. We propose a Kähler potential derivative of complex Gaussian mixtures that acts as a rough proxy to the Fisher information metric while still being faithful to the underlying Kähler geometry. Computation of the metric via this potential is efficient, and through our potential, valid as a plurisubharmonic (PSH) function, large scale computational burden of automatic differentiation is displaced to small scale. Our methods leverage the law of total covariance to bridge behavior between our potential and the Fisher metric. We show that we can regularize the latent space with decoder geometry, and that we can sample in accordance with a weighted complex volume element. We demonstrate these strategies, at the exchange of sample variation, yield consistently smoother representations and fewer semantic outliers.

Method: DDRL uses forward KL divergence to anchor the policy to an off-policy data distribution, combining reward maximization with diffusion loss minimization in a theoretically robust framework.

Result: Extensive experiments (over 1M GPU hours and 10K human evaluations) show DDRL significantly improves rewards while preventing reward hacking, achieving highest human preference scores on high-resolution video generation tasks.

Conclusion: DDRL establishes a robust and scalable paradigm for diffusion model post-training that effectively aligns generative models with human preferences without the common pitfalls of existing RL approaches.

Abstract: Aligning generative diffusion models with human preferences via reinforcement learning (RL) is critical yet challenging. Most existing algorithms are often vulnerable to reward hacking, such as quality degradation, over-stylization, or reduced diversity. Our analysis demonstrates that this can be attributed to the inherent limitations of their regularization, which provides unreliable penalties. We introduce Data-regularized Diffusion Reinforcement Learning (DDRL), a novel framework that uses the forward KL divergence to anchor the policy to an off-policy data distribution. Theoretically, DDRL enables robust, unbiased integration of RL with standard diffusion training. Empirically, this translates into a simple yet effective algorithm that combines reward maximization with diffusion loss minimization. With over a million GPU hours of experiments and ten thousand double-blind human evaluations, we demonstrate on high-resolution video generation tasks that DDRL significantly improves rewards while alleviating the reward hacking seen in baselines, achieving the highest human preference and establishing a robust and scalable paradigm for diffusion post-training.

Method: The paper evaluates limitations of existing interference prediction approaches, analyzing coarse-grained methods and static prediction models.

Result: Coarse-grained methods lead to noticeable deviations in prediction accuracy, and static models degrade considerably under changing workloads.

Conclusion: Existing interference prediction approaches have significant limitations that restrict their usefulness for scheduling in ML inference serving systems.

Abstract: Machine learning (ML) inference serving systems can schedule requests to improve GPU utilization and to meet service level objectives (SLOs) or deadlines. However, improving GPU utilization may compromise latency-sensitive scheduling, as concurrent tasks contend for GPU resources and thereby introduce interference. Given that interference effects introduce unpredictability in scheduling, neglecting them may compromise SLO or deadline satisfaction. Nevertheless, existing interference prediction approaches remain limited in several respects, which may restrict their usefulness for scheduling. First, they are often coarse-grained, which ignores runtime co-location dynamics and thus restricts their accuracy in interference prediction. Second, they tend to use a static prediction model, which may not effectively cope with different workload characteristics. In this paper, we evaluate the potential limitations of existing interference prediction approaches, finding that coarse-grained methods can lead to noticeable deviations in prediction accuracy and that static models degrade considerably under changing workloads.

Method: Propose CAse Prompting (CAP) - a training-free, clinically adaptive prompting framework that integrates conventional debiasing prompts with case-based reasoning. CAP guides models to learn from similar historical misprediction cases and their correct outcomes to correct biased reasoning patterns. Includes multi-dimensional bias assessment for comprehensive diagnosis.

Result: On MIMIC-IV dataset: AUROC increased from 0.806 to 0.873, AUPRC from 0.497 to 0.694. Reduced sex- and race-related disparities by over 90%. Feature reliance analysis shows highly consistent attention patterns across demographic groups (similarity scores >0.98).

Conclusion: LLMs exhibit measurable bias in ICU mortality prediction. CAP effectively co-optimizes fairness and performance without retraining, offering a transferable paradigm for equitable clinical decision support.

Abstract: Accurate mortality risk prediction for intensive care unit (ICU) patients is essential for clinical decision-making. Although large language models (LLMs) show promise in predicting outcomes from structured medical data, their predictions may exhibit demographic biases related to sex, age, and race, limiting their trustworthy use in clinical practice. Existing debiasing methods often reduce predictive performance, making it difficult to jointly optimize fairness and accuracy. In this study, we systematically examine bias in LLM-based ICU mortality prediction and propose a training-free, clinically adaptive prompting framework to simultaneously improve fairness and performance. We first develop a multi-dimensional bias assessment scheme for comprehensive model diagnosis. Building on this analysis, we introduce CAse Prompting (CAP), a novel prompting framework that integrates conventional debiasing prompts with case-based reasoning. CAP guides the model to learn from similar historical misprediction cases and their correct outcomes, enabling correction of biased reasoning patterns. Experiments on the MIMIC-IV dataset show that CAP substantially improves both predictive accuracy and fairness. CAP increases AUROC from 0.806 to 0.873 and AUPRC from 0.497 to 0.694, while reducing sex- and race-related disparities by over 90%. Feature reliance analysis further indicates highly consistent attention patterns across demographic groups, with similarity scores exceeding 0.98. These results demonstrate that LLMs exhibit measurable bias in ICU mortality prediction, and that a carefully designed prompting framework can effectively co-optimize fairness and performance without retraining, offering a transferable paradigm for equitable clinical decision support.

Method: Replaces standard attention with GALE (Geometry-Aware Physics Attention), coupling physics-aware self-attention on learned state slices with cross-attention to shared geometry/global/boundary-condition context computed from multi-scale ball queries (inspired by DoMINO). Persistently projects geometry, global and boundary condition parameters into physical state spaces to anchor latent computations to domain structure.

Result: Benchmarked on DrivAerML, Luminary SHIFT-SUV, and Luminary SHIFT-Wing datasets, GeoTransolver delivers better accuracy (drag/lift R2 and Relative L1 errors), improved robustness to geometry/regime shifts, and favorable data efficiency compared to Domino, Transolver, and AB-UPT.

Conclusion: GeoTransolver advances operator learning for high-fidelity surrogate modeling by unifying multiscale geometry-aware context with physics-based attention in a scalable transformer, enabling accurate modeling across complex, irregular domains and non-linear physical regimes.

Abstract: We present GeoTransolver, a Multiscale Geometry-Aware Physics Attention Transformer for CAE that replaces standard attention with GALE, coupling physics-aware self-attention on learned state slices with cross-attention to a shared geometry/global/boundary-condition context computed from multi-scale ball queries (inspired by DoMINO) and reused in every block. Implemented and released in NVIDIA PhysicsNeMo, GeoTransolver persistently projects geometry, global and boundary condition parameters into physical state spaces to anchor latent computations to domain structure and operating regimes. We benchmark GeoTransolver on DrivAerML, Luminary SHIFT-SUV, and Luminary SHIFT-Wing, comparing against Domino, Transolver (as released in PhysicsNeMo), and literature-reported AB-UPT, and evaluate drag/lift R2 and Relative L1 errors for field variables. GeoTransolver delivers better accuracy, improved robustness to geometry/regime shifts, and favorable data efficiency; we include ablations on DrivAerML and qualitative results such as contour plots and design trends for the best GeoTransolver models. By unifying multiscale geometry-aware context with physics-based attention in a scalable transformer, GeoTransolver advances operator learning for high-fidelity surrogate modeling across complex, irregular domains and non-linear physical regimes.

Method: Introduces a novel decentralized framework for optimal joint action selection that explicitly accounts for belief inconsistencies, provides probabilistic guarantees for action consistency and performance relative to open-loop multi-agent POMDP, and selectively triggers communication only when needed.

Result: Simulation results show the approach outperforms state-of-the-art algorithms.

Conclusion: The framework successfully addresses the critical challenge of belief inconsistencies in multi-agent decision-making while maintaining probabilistic guarantees and communication efficiency.

Abstract: Multi-agent decision-making under uncertainty is fundamental for effective and safe autonomous operation. In many real-world scenarios, each agent maintains its own belief over the environment and must plan actions accordingly. However, most existing approaches assume that all agents have identical beliefs at planning time, implying these beliefs are conditioned on the same data. Such an assumption is often impractical due to limited communication. In reality, agents frequently operate with inconsistent beliefs, which can lead to poor coordination and suboptimal, potentially unsafe, performance. In this paper, we address this critical challenge by introducing a novel decentralized framework for optimal joint action selection that explicitly accounts for belief inconsistencies. Our approach provides probabilistic guarantees for both action consistency and performance with respect to open-loop multi-agent POMDP (which assumes all data is always communicated), and selectively triggers communication only when needed. Furthermore, we address another key aspect of whether, given a chosen joint action, the agents should share data to improve expected performance in inference. Simulation results show our approach outperforms state-of-the-art algorithms.

Method: Three key innovations: 1) On-chain DAO governance for transparent coordination and immutable logging; 2) ZKP mechanism for off-chain Shapley-based contribution measurement; 3) Hybrid on-chain/off-chain architecture that verifies ZKP-validated contributions on-chain with minimal computational overhead.

Result: Experimental implementation using crypto trading tasks shows DAO-Agent achieves up to 99.9% reduction in verification gas costs compared to naive on-chain alternatives, with constant-time verification complexity that remains stable as coalition size increases.

Conclusion: DAO-Agent establishes a scalable foundation for agent coordination in decentralized environments by enabling auditable task execution and fair incentive distribution while preserving strategic privacy and minimizing on-chain costs.

Abstract: Autonomous Large Language Model (LLM)-based multi-agent systems have emerged as a promising paradigm for facilitating cross-application and cross-organization collaborations. These autonomous agents often operate in trustless environments, where centralized coordination faces significant challenges, such as the inability to ensure transparent contribution measurement and equitable incentive distribution. While blockchain is frequently proposed as a decentralized coordination platform, it inherently introduces high on-chain computation costs and risks exposing sensitive execution information of the agents. Consequently, the core challenge lies in enabling auditable task execution and fair incentive distribution for autonomous LLM agents in trustless environments, while simultaneously preserving their strategic privacy and minimizing on-chain costs. To address this challenge, we propose DAO-Agent, a novel framework that integrates three key technical innovations: (1) an on-chain decentralized autonomous organization (DAO) governance mechanism for transparent coordination and immutable logging; (2) a ZKP mechanism approach that enables Shapley-based contribution measurement off-chain, and (3) a hybrid on-chain/off-chain architecture that verifies ZKP-validated contribution measurements on-chain with minimal computational overhead. We implement DAO-Agent and conduct end-to-end experiments using a crypto trading task as a case study. Experimental results demonstrate that DAO-Agent achieves up to 99.9% reduction in verification gas costs compared to naive on-chain alternatives, with constant-time verification complexity that remains stable as coalition size increases, thereby establishing a scalable foundation for agent coordination in decentralized environments.

Method: AgentReuse leverages semantic similarities and differences among requests, using intent classification to evaluate request similarities and enable plan reuse. The system identifies similar requests to reuse previously generated plans instead of generating new ones from scratch.

Result: AgentReuse achieves 93% effective plan reuse rate, F1 score of 0.9718, and accuracy of 0.9459 in evaluating request similarities. It reduces latency by 93.12% compared to baselines without reuse mechanisms.

Conclusion: AgentReuse effectively addresses the latency problem in LLM-driven agents by enabling plan reuse through semantic similarity analysis and intent classification, significantly improving user experience while maintaining high accuracy in request matching.

Abstract: Integrating large language models (LLMs) into personal assistants, like Xiao Ai and Blue Heart V, effectively enhances their ability to interact with humans, solve complex tasks, and manage IoT devices. Such assistants are also termed LLM-driven agents. Upon receiving user requests, the LLM-driven agent generates plans using an LLM, executes these plans through various tools, and then returns the response to the user. During this process, the latency for generating a plan with an LLM can reach tens of seconds, significantly degrading user experience. Real-world dataset analysis shows that about 30% of the requests received by LLM-driven agents are identical or similar, which allows the reuse of previously generated plans to reduce latency. However, it is difficult to accurately define the similarity between the request texts received by the LLM-driven agent through directly evaluating the original request texts. Moreover, the diverse expressions of natural language and the unstructured format of plan texts make implementing plan reuse challenging. To address these issues, we present and implement a plan reuse mechanism for LLM-driven agents called AgentReuse. AgentReuse leverages the similarities and differences among requests’ semantics and uses intent classification to evaluate the similarities between requests and enable the reuse of plans. Experimental results based on a real-world dataset demonstrate that AgentReuse achieves a 93% effective plan reuse rate, an F1 score of 0.9718, and an accuracy of 0.9459 in evaluating request similarities, reducing latency by 93.12% compared with baselines without using the reuse mechanism.

Method: Develop computational foundations formalizing two coopetition dimensions: (1) interdependence grounded in i* structural dependency analysis, translating depender-dependee-dependum relationships into quantitative interdependence coefficients; (2) complementarity formalized using Brandenburger and Nalebuff’s Added Value concept with synergistic value creation modeling. Integrates structural dependencies with bargaining power in value appropriation and introduces game-theoretic formulation where Nash Equilibrium incorporates structural interdependence.

Result: Validation across 22,000+ experimental trials shows logarithmic specifications achieve 58/60 alignment score vs. power functions’ 46/60 when compared to historical S-LCD data. Logarithmic specifications produce realistic 41% cooperation increases aligning with documented patterns, while power functions produce unrealistic 166% increases. Statistical significance confirmed at p < 0.001 with Cohen’s d > 9.

Conclusion: The framework successfully bridges qualitative conceptual modeling with quantitative game theory for coopetition analysis, providing a rigorous computational foundation that preserves contextual richness while enabling dynamic trade-off analysis, validated through real-world joint venture case study.

Abstract: Coopetition refers to simultaneous cooperation and competition among actors wherein actors ‘cooperate to grow the pie and compete to split it up.’ Modern socio-technical systems are characterized by strategic coopetition wherein actors concomitantly cooperate to create value and compete to capture it. While conceptual modeling languages such as i* provide rich qualitative representations of strategic dependencies, they lack mechanisms for quantitative analysis of dynamic trade-offs. Conversely, classical game theory offers mathematical rigor but strips away contextual richness. This report bridges this gap by developing computational foundations that formalize two critical dimensions of coopetition: interdependence and complementarity. We ground interdependence in i* structural dependency analysis, translating depender-dependee-dependum relationships into quantitative interdependence coefficients via a structured translation framework. We formalize complementarity following Brandenburger and Nalebuff’s Added Value concept, modeling synergistic value creation with validated parameterization. We integrate structural dependencies with bargaining power in value appropriation and introduce a game-theoretic formulation where Nash Equilibrium incorporates structural interdependence. Validation combines over 22,000 experimental trials across power and logarithmic specifications with the Samsung-Sony S-LCD joint venture (2004-2011). Under strict historical alignment scoring, logarithmic specifications achieve 58/60 compared to power functions (46/60), producing realistic 41% cooperation increases aligning with documented S-LCD patterns while power functions produce 166% increases exceeding realistic bounds. Statistical significance confirmed at p < 0.001, Cohen’s d > 9.

Method: Two-stage decoder-only LM: Stage-1 predicts coarse semantic tokens, Stage-2 generates fine acoustic tokens. Uses continuous SSL/codec embeddings for richer context. Employs Frozen-LM Conditioning to reduce exposure bias and DPO for human perceptual alignment.

Result: Experiments on Libri2Mix show GenTSE surpasses previous LM-based systems in speech quality, intelligibility, and speaker consistency.

Conclusion: Separating semantics and acoustics stabilizes decoding and yields more faithful, content-aligned target speech, with training strategies that bridge the gap between training and inference.

Abstract: Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech. We present GenTSE, a two-stage decoder-only generative LM approach for TSE: Stage-1 predicts coarse semantic tokens, and Stage-2 generates fine acoustic tokens. Separating semantics and acoustics stabilizes decoding and yields more faithful, content-aligned target speech. Both stages use continuous SSL or codec embeddings, offering richer context than discretized-prompt methods. To reduce exposure bias, we employ a Frozen-LM Conditioning training strategy that conditions the LMs on predicted tokens from earlier checkpoints to reduce the gap between teacher-forcing training and autoregressive inference. We further employ DPO to better align outputs with human perceptual preferences. Experiments on Libri2Mix show that GenTSE surpasses previous LM-based systems in speech quality, intelligibility, and speaker consistency.

Method: Two-component architecture: 1) Encoder-Decoder Attractor (EDA) network that automatically infers source count and acoustic clues for sound separation, and 2) multi-modal fusion network that interprets diverse user clues (acoustic, semantic, visual) for target sound extraction. Joint training with cross-task consistency constraints creates a unified latent space.

Result: Remarkable performance in both tasks: 1.4 dB SDR improvement in sound separation compared to baseline, and 86% accuracy in target sound extraction.

Conclusion: The proposed unified framework successfully bridges sound separation and target sound extraction paradigms, enabling adaptive operation in either fully autonomous or clue-driven modes while achieving state-of-the-art performance.

Abstract: Sound separation (SS) and target sound extraction (TSE) are fundamental techniques for addressing complex acoustic scenarios. While existing SS methods struggle with determining the unknown number of sound sources, TSE approaches require precisely specified clues to achieve optimal performance. This paper proposes a unified framework that synergistically combines SS and TSE to overcome their individual limitations. Our architecture employs two complementary components: 1) An Encoder-Decoder Attractor (EDA) network that automatically infers both the source count and corresponding acoustic clues for SS, and 2) A multi-modal fusion network that precisely interprets diverse user-provided clues (acoustic, semantic, or visual) for TSE. Through joint training with cross-task consistency constraints, we establish a unified latent space that bridges both paradigms. During inference, the system adaptively operates in either fully autonomous SS mode or clue-driven TSE mode. Experiments demonstrate remarkable performance in both tasks, with notable improvements of 1.4 dB SDR improvement in SS compared to baseline and 86% TSE accuracy.

Method: Enhanced ASCHOPLEX integrated within Dafne (Deep Anatomical Federated Network) framework for federated incremental learning. Comparative evaluation of federated approach vs conventional fine-tuning on 2,284 subjects from 5 independent MRI datasets including Multiple Sclerosis patients and healthy controls.

Result: Conventional fine-tuning works well on homogeneous data but has limited generalizability with high variability. Federated incremental learning consistently achieves higher generalizability and more stable performance across diverse acquisition settings.

Conclusion: Federated incremental learning provides a robust alternative to conventional fine-tuning for Choroid Plexus segmentation, improving model generalizability across heterogeneous MRI datasets and acquisition conditions.

Abstract: The Choroid Plexus (ChP) is a highly vascularized brain structure that plays a critical role in several physiological processes. ASCHOPLEX, a deep learning-based segmentation toolbox with an integrated fine-tuning stage, provides accurate ChP delineations on non-contrast-enhanced T1-weighted MRI scans; however, its performance is hindered by inter-dataset variability. This study introduces the first federated incremental learning approach for automated ChP segmentation from 3D T1-weighted brain MRI, by integrating an enhanced version of ASCHOPLEX within the Dafne (Deep Anatomical Federated Network) framework. A comparative evaluation is conducted to assess whether federated incremental learning through Dafne improves model generalizability across heterogeneous imaging conditions, relative to the conventional fine-tuning strategy employed by standalone ASCHOPLEX. The experimental cohort comprises 2,284 subjects, including individuals with Multiple Sclerosis as well as healthy controls, collected from five independent MRI datasets. Results indicate that the fine-tuning strategy provides high performance on homogeneous data (e.g., same MRI sequence, same cohort of subjects), but limited generalizability when the data variability is high (e.g., multiple MRI sequences, multiple and new cohorts of subjects). By contrast, the federated incremental learning variant of ASCHOPLEX constitutes a robust alternative consistently achieving higher generalizability and more stable performance across diverse acquisition settings.

Method: An instance-specific overfitted paradigm that directly optimizes channel symbols and a lightweight neural decoder for each source, using as few as 607 model parameters and 641 multiplications per pixel.

Result: Achieves around 1000x lower decoding complexity than alternatives, obtains state-of-the-art results in high SNR regimes, and enables one-time offline encoding supporting multiple online decoding for streaming scenarios.

Conclusion: Implicit-JSCC shows promise for future communication systems, particularly streaming applications, by providing an efficient, low-complexity, modality-agnostic solution that inherently addresses source generalizability.

Abstract: This paper introduces Implicit-JSCC, a novel overfitted joint source-channel coding paradigm that directly optimizes channel symbols and a lightweight neural decoder for each source. This instance-specific strategy eliminates the need for training datasets or pre-trained models, enabling a storage-free, modality-agnostic solution. As a low-complexity alternative, Implicit-JSCC achieves efficient image transmission with around 1000x lower decoding complexity, using as few as 607 model parameters and 641 multiplications per pixel. This overfitted design inherently addresses source generalizability and achieves state-of-the-art results in the high SNR regimes, underscoring its promise for future communication systems, especially streaming scenarios where one-time offline encoding supports multiple online decoding.

Method: Proposed SL-Net, a single-shot deep learning model that removes solar loading transients from thermal facial images. Created and open-sourced a diverse dataset of 100 subjects with co-registered RGB-thermal images, IRT measurements, and skin tone data.

Result: Forehead skin temperature increases by 2.00°C after solar loading. SL-Net reduces this error by 68% to 0.64°C. The model eliminates skin tone bias in IRT performance that solar loading introduces.

Conclusion: Machine learning can correct complex thermal perturbations like solar loading, enabling robust and equitable human thermography without requiring lengthy reacclimation periods.

Abstract: Widely deployed for fever detection, infrared thermometers (IRTs) enable rapid non-contact measurement of core body temperature but are inaccurate in unconstrained environments when skin temperature is transient. In this work, we present the first study on the effect of solar loading–solar radiation-induced elevation of skin but not core temperature–on IRT performance. Solar loading causes poor specificity in IRT fever detection, and the standard procedure is to reacclimate subjects for up to 30 minutes before IRT measurement. In contrast, we propose a single-shot deep learning model that removes solar loading transients from thermal facial images, allowing accurate IRT operation in solar loaded conditions. Forehead skin temperature increases by 2.00°C after solar loading, and our deep learning model, SL-Net, reduces this error by 68% to 0.64°C. We show that the solar loading effect depends on skin tone, introducing inequity in IRT performance, while SL-Net is unbiased. We open source a diverse dataset of 100 subjects with co-registered RGB-thermal images, and IRT and skin tone measurements. Our work shows that it is possible to use machine learning to correct complex thermal perturbations to enable robust and equitable human thermography.

Method: V-Rex introduces ReSV, a training-free dynamic KV cache retrieval algorithm using temporal and spatial similarity-based token clustering, plus a hardware accelerator with dynamic KV cache retrieval engine (DRE) featuring bit-level and early-exit computing units.

Result: Achieves 3.9-8.3 FPS real-time inference on edge devices with negligible accuracy loss, 1.9-19.7x speedup and 3.1-18.5x energy efficiency improvements over AGX Orin GPU, while DRE only uses 2.2% power and 2.0% area.

Conclusion: First comprehensive solution tackling KV cache retrieval across algorithms and hardware, enabling real-time streaming video LLM inference on resource-constrained edge devices.

Abstract: Streaming video large language models (LLMs) are increasingly used for real-time multimodal tasks such as video captioning, question answering, conversational agents, and augmented reality. However, these models face fundamental memory and computational challenges because their key-value (KV) caches grow substantially with continuous streaming video input. This process requires an iterative prefill stage, which is a unique feature of streaming video LLMs. Due to its iterative prefill stage, it suffers from significant limitations, including extensive computation, substantial data transfer, and degradation in accuracy. Crucially, this issue is exacerbated for edge deployment, which is the primary target for these models. In this work, we propose V-Rex, the first software-hardware co-designed accelerator that comprehensively addresses both algorithmic and hardware bottlenecks in streaming video LLM inference. At its core, V-Rex introduces ReSV, a training-free dynamic KV cache retrieval algorithm. ReSV exploits temporal and spatial similarity-based token clustering to reduce excessive KV cache memory across video frames. To fully realize these algorithmic benefits, V-Rex offers a compact, low-latency hardware accelerator with a dynamic KV cache retrieval engine (DRE), featuring bit-level and early-exit based computing units. V-Rex achieves unprecedented real-time of 3.9-8.3 FPS and energy-efficient streaming video LLM inference on edge deployment with negligible accuracy loss. While DRE only accounts for 2.2% power and 2.0% area, the system delivers 1.9-19.7x speedup and 3.1-18.5x energy efficiency improvements over AGX Orin GPU. This work is the first to comprehensively tackle KV cache retrieval across algorithms and hardware, enabling real-time streaming video LLM inference on resource-constrained edge devices.

Method: Created a multi-center breast MRI dataset from 6 clinical institutions across 5 European countries, comprising 741 examinations with malignant, benign, and non-lesion cases. Conducted baseline experiments using a transformer-based model.

Result: Developed a publicly available dataset with heterogeneous scanners, field strengths, and acquisition protocols reflecting real-world variability. Provided benchmark experiments to illustrate dataset use and establish reference performance for future comparisons.

Conclusion: This publicly available breast MRI dataset addresses a critical gap in AI development for breast cancer detection and provides a foundation for future methodological advancements and comparisons.

Abstract: Early detection of breast cancer is critical for improving patient outcomes. While mammography remains the primary screening modality, magnetic resonance imaging (MRI) is increasingly recommended as a supplemental tool for women with dense breast tissue and those at elevated risk. However, the acquisition and interpretation of multiparametric breast MRI are time-consuming and require specialized expertise, limiting scalability in clinical practice. Artificial intelligence (AI) methods have shown promise in supporting breast MRI interpretation, but their development is hindered by the limited availability of large, diverse, and publicly accessible datasets. To address this gap, we present a publicly available, multi-center breast MRI dataset collected across six clinical institutions in five European countries. The dataset comprises 741 examinations from women undergoing screening or diagnostic breast MRI and includes malignant, benign, and non-lesion cases. Data were acquired using heterogeneous scanners, field strengths, and acquisition protocols, reflecting real-world clinical variability. In addition, we report baseline benchmark experiments using a transformer-based model to illustrate potential use cases of the dataset and to provide reference performance for future methodological comparisons.