Method: SBP first learns a model of relations between documents from the pretraining dataset, then uses it to synthesize a vast new corpus for joint training. It abstracts core concepts from seed material and creates new narrations.

Result: SBP consistently improves upon repetition baselines and achieves significant performance improvement comparable to using 20x more unique data. The synthesized documents go beyond paraphrasing to create genuinely new content.

Conclusion: SBP provides strong empirical performance gains and has a natural Bayesian interpretation where the synthesizer learns to abstract latent concepts shared between related documents.

Abstract: We introduce Synthetic Bootstrapped Pretraining (SBP), a language model (LM) pretraining procedure that first learns a model of relations between documents from the pretraining dataset and then leverages it to synthesize a vast new corpus for joint training. While the standard pretraining teaches LMs to learn causal correlations among tokens within a single document, it is not designed to efficiently model the rich, learnable inter-document correlations that can potentially lead to better performance. We validate SBP by designing a compute-matched pretraining setup and pretrain a 3B-parameter model on up to 1T tokens from scratch. We find SBP consistently improves upon a strong repetition baseline and delivers a significant fraction of performance improvement attainable by an oracle upper bound with access to 20x more unique data. Qualitative analysis reveals that the synthesized documents go beyond mere paraphrases – SBP first abstracts a core concept from the seed material and then crafts a new narration on top of it. Besides strong empirical performance, SBP admits a natural Bayesian interpretation: the synthesizer implicitly learns to abstract the latent concepts shared between related documents.

Method: Evaluated three tokenization algorithms (Byte-Pair Encoding, WordPiece, SentencePiece) using metrics including Subword Fertility, Proportion of Continued Words, Normalized Sequence Length, and execution time to assess their suitability for Dzongkha.

Result: All three algorithms showed potential, but SentencePiece (Unigram) was identified as the most effective tokenization method for Dzongkha based on the performance metrics.

Conclusion: SentencePiece is the optimal choice for Dzongkha tokenization, highlighting the need for tailored approaches for low-resource languages and enabling future development of Dzongkha Large Language Models.

Abstract: Large Language Models (LLMs) are gaining popularity and improving rapidly. Tokenizers are crucial components of natural language processing, especially for LLMs. Tokenizers break down input text into tokens that models can easily process while ensuring the text is accurately represented, capturing its meaning and structure. Effective tokenizers enhance the capabilities of LLMs by improving a model’s understanding of context and semantics, ultimately leading to better performance in various downstream tasks, such as translation, classification, sentiment analysis, and text generation. Most pre-trained tokenizers are suitable for high-resource languages like English but perform poorly for low-resource languages. Dzongkha, Bhutan’s national language spoken by around seven hundred thousand people, is a low-resource language, and its linguistic complexity poses unique NLP challenges. Despite some progress, significant research in Dzongkha NLP is lacking, particularly in tokenization. This study evaluates the training and performance of three common tokenization algorithms in comparison to other popular methods. Specifically, Byte-Pair Encoding (BPE), WordPiece, and SentencePiece (Unigram) were evaluated for their suitability for Dzongkha. Performance was assessed using metrics like Subword Fertility, Proportion of Continued Words, Normalized Sequence Length, and execution time. The results show that while all three algorithms demonstrate potential, SentencePiece is the most effective for Dzongkha tokenization, paving the way for further NLP advancements. This underscores the need for tailored approaches for low-resource languages and ongoing research. In this study, we presented three tokenization algorithms for Dzongkha, paving the way for building Dzongkha Large Language Models.

Method: Adopts a red-teaming approach to systematically probe LLM vulnerabilities across three real-world scenarios: conversation, question-answering, and content composition, using a novel dataset covering Singlish, Chinese, Malay, and Tamil.

Result: Extensive experiments with state-of-the-art multilingual LLMs uncover critical gaps in their safety guardrails.

Conclusion: The work provides actionable insights into cultural sensitivity and toxicity mitigation, laying the foundation for safer and more inclusive AI systems in linguistically diverse environments.

Abstract: The advancement of Large Language Models (LLMs) has transformed natural language processing; however, their safety mechanisms remain under-explored in low-resource, multilingual settings. Here, we aim to bridge this gap. In particular, we introduce \textsf{SGToxicGuard}, a novel dataset and evaluation framework for benchmarking LLM safety in Singapore’s diverse linguistic context, including Singlish, Chinese, Malay, and Tamil. SGToxicGuard adopts a red-teaming approach to systematically probe LLM vulnerabilities in three real-world scenarios: \textit{conversation}, \textit{question-answering}, and \textit{content composition}. We conduct extensive experiments with state-of-the-art multilingual LLMs, and the results uncover critical gaps in their safety guardrails. By offering actionable insights into cultural sensitivity and toxicity mitigation, we lay the foundation for safer and more inclusive AI systems in linguistically diverse environments.\footnote{Link to the dataset: https://github.com/Social-AI-Studio/SGToxicGuard.} \textcolor{red}{Disclaimer: This paper contains sensitive content that may be disturbing to some readers.}

Method: Constructed minimal pairs of factually equivalent German claims differing only in political connotation (euphemisms/dysphemisms). Evaluated six LLMs’ consistency in truthfulness classification and tested whether explicit calls for objectivism in prompts mitigate bias.

Result: Judgmental words significantly influence truthfulness assessment more than political leaning. Few models show political bias tendencies, and explicit objectivism prompts don’t effectively mitigate bias.

Conclusion: Political bias in LLMs for fact-checking is more influenced by judgmental language than political leaning, and current mitigation approaches like explicit objectivism prompts are insufficient.

Abstract: Large Language Models are increasingly used in applications requiring objective assessment, which could be compromised by political bias. Many studies found preferences for left-leaning positions in LLMs, but downstream effects on tasks like fact-checking remain underexplored. In this study, we systematically investigate political bias through exchanging words with euphemisms or dysphemisms in German claims. We construct minimal pairs of factually equivalent claims that differ in political connotation, to assess the consistency of LLMs in classifying them as true or false. We evaluate six LLMs and find that, more than political leaning, the presence of judgmental words significantly influences truthfulness assessment. While a few models show tendencies of political bias, this is not mitigated by explicitly calling for objectivism in prompts.

Method: Proposes Approximate Question-side Effect (AQE) to quantify question-awareness contribution, and introduces SCAO (Semantic Compression by Answering in One word) to enhance model-side signals and reduce reliance on question-side cues.

Result: Analysis reveals much reported success in hallucination prediction stems from exploiting superficial question patterns. SCAO achieves strong, consistent performance especially when question-side cues are reduced, demonstrating effectiveness in fostering genuine self-awareness.

Conclusion: Hallucination prediction performance should not be automatically interpreted as self-awareness, as question-side shortcuts play a significant role. SCAO provides a more reliable approach for developing genuine introspection capabilities in LLMs.

Abstract: Hallucination prediction in large language models (LLMs) is often interpreted as a sign of self-awareness. However, we argue that such performance can arise from question-side shortcuts rather than true model-side introspection. To disentangle these factors, we propose the Approximate Question-side Effect (AQE), which quantifies the contribution of question-awareness. Our analysis across multiple datasets reveals that much of the reported success stems from exploiting superficial patterns in questions. We further introduce SCAO (Semantic Compression by Answering in One word), a method that enhances the use of model-side signals. Experiments show that SCAO achieves strong and consistent performance, particularly in settings with reduced question-side cues, highlighting its effectiveness in fostering genuine self-awareness in LLMs.

Method: HERO combines predictions from length-specialist models trained with Subcategory Guidance, which encourages separation of easily confused categories (e.g., different source languages) to boost performance.

Result: Extensive experiments across 5 LLMs and 6 domains show HERO outperforms state-of-the-art by 2.5-3 mAP on average.

Conclusion: HERO provides a robust framework for fine-grained machine-influenced text detection that handles varying text lengths and improves classification of nuanced LLM usage scenarios.

Abstract: Large Language Model (LLMs) can be used to write or modify documents, presenting a challenge for understanding the intent behind their use. For example, benign uses may involve using LLM on a human-written document to improve its grammar or to translate it into another language. However, a document entirely produced by a LLM may be more likely to be used to spread misinformation than simple translation (\eg, from use by malicious actors or simply by hallucinating). Prior works in Machine Generated Text (MGT) detection mostly focus on simply identifying whether a document was human or machine written, ignoring these fine-grained uses. In this paper, we introduce a HiErarchical, length-RObust machine-influenced text detector (HERO), which learns to separate text samples of varying lengths from four primary types: human-written, machine-generated, machine-polished, and machine-translated. HERO accomplishes this by combining predictions from length-specialist models that have been trained with Subcategory Guidance. Specifically, for categories that are easily confused (\eg, different source languages), our Subcategory Guidance module encourages separation of the fine-grained categories, boosting performance. Extensive experiments across five LLMs and six domains demonstrate the benefits of our HERO, outperforming the state-of-the-art by 2.5-3 mAP on average.

Method: The framework distinguishes core semantics from spurious contexts using counterfactual examples for training-stage debiasing, and employs a Mixture-of-Experts architecture with dynamic routing to selectively engage modality-specific debiasing experts.

Result: Empirical evaluation on multimodal sarcasm detection and sentiment analysis tasks shows the framework significantly outperforms unimodal debiasing strategies and existing state-of-the-art models.

Conclusion: The proposed causal mediation-based debiasing framework effectively addresses superficial correlation bias in MLLMs, enhancing their robustness and generalization capabilities in multimodal reasoning.

Abstract: Multimodal Large Language Models (MLLMs) have shown substantial capabilities in integrating visual and textual information, yet frequently rely on spurious correlations, undermining their robustness and generalization in complex multimodal reasoning tasks. This paper addresses the critical challenge of superficial correlation bias in MLLMs through a novel causal mediation-based debiasing framework. Specially, we distinguishing core semantics from spurious textual and visual contexts via counterfactual examples to activate training-stage debiasing and employ a Mixture-of-Experts (MoE) architecture with dynamic routing to selectively engages modality-specific debiasing experts. Empirical evaluation on multimodal sarcasm detection and sentiment analysis tasks demonstrates that our framework significantly surpasses unimodal debiasing strategies and existing state-of-the-art models.

Method: Collect large-scale spontaneous high-quality Wolof speech data, continue pretraining HuBERT on this dataset, integrate the speech encoder into a Wolof LLM, and explore multi-step Chain-of-Thought reasoning before transcription/translation.

Result: The Speech LLM outperforms base models and African-centric models on ASR, improves speech recognition, and performs well in speech translation tasks.

Conclusion: The approach successfully creates the first Speech LLM for Wolof, demonstrating improved performance in speech tasks, with models and code to be openly shared to benefit underrepresented language communities.

Abstract: We present our journey in training a speech language model for Wolof, an underrepresented language spoken in West Africa, and share key insights. We first emphasize the importance of collecting large-scale, spontaneous, high-quality speech data, and show that continued pretraining HuBERT on this dataset outperforms both the base model and African-centric models on ASR. We then integrate this speech encoder into a Wolof LLM to train the first Speech LLM for this language, extending its capabilities to tasks such as speech translation. Furthermore, we explore training the Speech LLM to perform multi-step Chain-of-Thought before transcribing or translating. Our results show that the Speech LLM not only improves speech recognition but also performs well in speech translation. The models and the code will be openly shared.

Method: Systematically evaluate LLMs and multimodal LLMs on English (MUStARD++) and Chinese (MCSD 1.0) datasets using zero-shot, few-shot, and LoRA fine-tuning settings, plus feature encoding with collaborative gating fusion module.

Result: Audio-based models achieve strongest unimodal performance; text-audio and audio-vision combinations outperform unimodal and trimodal models; MLLMs like Qwen-Omni show competitive zero-shot and fine-tuned performance.

Conclusion: MLLMs show potential for cross-lingual, audio-visual-textual sarcasm understanding, with audio playing a crucial role in sarcasm detection.

Abstract: Sarcasm detection remains a challenge in natural language understanding, as sarcastic intent often relies on subtle cross-modal cues spanning text, speech, and vision. While prior work has primarily focused on textual or visual-textual sarcasm, comprehensive audio-visual-textual sarcasm understanding remains underexplored. In this paper, we systematically evaluate large language models (LLMs) and multimodal LLMs for sarcasm detection on English (MUStARD++) and Chinese (MCSD 1.0) in zero-shot, few-shot, and LoRA fine-tuning settings. In addition to direct classification, we explore models as feature encoders, integrating their representations through a collaborative gating fusion module. Experimental results show that audio-based models achieve the strongest unimodal performance, while text-audio and audio-vision combinations outperform unimodal and trimodal models. Furthermore, MLLMs such as Qwen-Omni show competitive zero-shot and fine-tuned performance. Our findings highlight the potential of MLLMs for cross-lingual, audio-visual-textual sarcasm understanding.

Method: Three text-based augmentation techniques: gloss-based replacement, random replacement, and LLM-based text generation, followed by Text-to-Speech (TTS) conversion to create synthetic audio data. The methods use only original annotated data without external resources.

Result: Fine-tuning Wav2Vec2-XLSR-53 on combined original and synthetic data achieved significant performance gains, including 14.3% absolute WER reduction for Nashta. Methods proved effective across all four low-resource languages (Vatlongos, Nashta, Shinekhen Buryat, Kakabe) and showed utility for high-resource languages like English.

Conclusion: The proposed self-contained data augmentation methods are broadly applicable and effective for improving ASR performance in low-resource scenarios, demonstrating their potential for language preservation and resource-constrained ASR development.

Abstract: This paper introduces three self-contained data augmentation methods for low-resource Automatic Speech Recognition (ASR). Our techniques first generate novel text–using gloss-based replacement, random replacement, or an LLM-based approach–and then apply Text-to-Speech (TTS) to produce synthetic audio. We apply these methods, which leverage only the original annotated data, to four languages with extremely limited resources (Vatlongos, Nashta, Shinekhen Buryat, and Kakabe). Fine-tuning a pretrained Wav2Vec2-XLSR-53 model on a combination of the original audio and generated synthetic data yields significant performance gains, including a 14.3% absolute WER reduction for Nashta. The methods prove effective across all four low-resource languages and also show utility for high-resource languages like English, demonstrating their broad applicability.

Method: The authors propose a novel uncertainty estimation framework that works post-hoc and is model-agnostic, along with a robust uncertainty estimation method designed to maintain valid uncertainty guarantees even under noisy conditions.

Result: Extensive experiments on QA tasks demonstrate the desired performance of the proposed methods.

Conclusion: The work successfully bridges the gap in uncertainty quantification for natural language explanations of LLMs, providing reliable confidence measures for explanations generated by complex language models.

Abstract: Large language models (LLMs) have shown strong capabilities, enabling concise, context-aware answers in question answering (QA) tasks. The lack of transparency in complex LLMs has inspired extensive research aimed at developing methods to explain large language behaviors. Among existing explanation methods, natural language explanations stand out due to their ability to explain LLMs in a self-explanatory manner and enable the understanding of model behaviors even when the models are closed-source. However, despite these promising advancements, there is no existing work studying how to provide valid uncertainty guarantees for these generated natural language explanations. Such uncertainty quantification is critical in understanding the confidence behind these explanations. Notably, generating valid uncertainty estimates for natural language explanations is particularly challenging due to the auto-regressive generation process of LLMs and the presence of noise in medical inquiries. To bridge this gap, in this work, we first propose a novel uncertainty estimation framework for these generated natural language explanations, which provides valid uncertainty guarantees in a post-hoc and model-agnostic manner. Additionally, we also design a novel robust uncertainty estimation method that maintains valid uncertainty guarantees even under noise. Extensive experiments on QA tasks demonstrate the desired performance of our methods.

Method: Fine-tuned PEGASUS and PEGASUS-X models on a medium-sized radiological reports dataset, evaluated different checkpoints with varying training data sizes, and monitored performance using lexical and semantic metrics on validation sets.

Result: PEGASUS showed phases related to epoch-wise double-descent or peak-drop-recovery behavior, while PEGASUS-X performance deteriorated when using larger checkpoints, indicating risks of overfitting with high-expressivity models on scarce data.

Conclusion: This work highlights the challenges of fine-tuning expressive models with limited training data in specialized domains and provides groundwork for developing more robust fine-tuning strategies for medical summarization models.

Abstract: Regardless of the rapid development of artificial intelligence, abstractive summarisation is still challenging for sensitive and data-restrictive domains like medicine. With the increasing number of imaging, the relevance of automated tools for complex medical text summarisation is expected to become highly relevant. In this paper, we investigated the adaptation via fine-tuning process of a non-domain-specific abstractive summarisation encoder-decoder model family, and gave insights to practitioners on how to avoid over- and underfitting. We used PEGASUS and PEGASUS-X, on a medium-sized radiological reports public dataset. For each model, we comprehensively evaluated two different checkpoints with varying sizes of the same training data. We monitored the models’ performances with lexical and semantic metrics during the training history on the fixed-size validation set. PEGASUS exhibited different phases, which can be related to epoch-wise double-descent, or peak-drop-recovery behaviour. For PEGASUS-X, we found that using a larger checkpoint led to a performance detriment. This work highlights the challenges and risks of fine-tuning models with high expressivity when dealing with scarce training data, and lays the groundwork for future investigations into more robust fine-tuning strategies for summarisation models in specialised domains.

Method: BiRQ uses a bilevel SSL framework where intermediate representations are discretized by a random-projection quantizer to produce enhanced labels, while anchoring labels from raw input stabilize training. It’s solved as a first-order bilevel optimization problem with differentiable Gumbel-softmax selection.

Result: BiRQ consistently improves over BEST-RQ while maintaining low complexity and computational efficiency, validated on various datasets including 960-hour LibriSpeech, 150-hour AMI meetings and 5,000-hour YODAS.

Conclusion: The proposed BiRQ framework eliminates the need for external label encoders, reduces memory cost, and enables iterative label refinement in an end-to-end fashion, achieving better performance than BEST-RQ with maintained efficiency.

Abstract: Speech is a rich signal, and labeled audio-text pairs are costly, making self-supervised learning essential for scalable representation learning. A core challenge in speech SSL is generating pseudo-labels that are both informative and efficient: strong labels, such as those used in HuBERT, improve downstream performance but rely on external encoders and multi-stage pipelines, while efficient methods like BEST-RQ achieve simplicity at the cost of weaker labels. We propose BiRQ, a bilevel SSL framework that combines the efficiency of BEST-RQ with the refinement benefits of HuBERT-style label enhancement. The key idea is to reuse part of the model itself as a pseudo-label generator: intermediate representations are discretized by a random-projection quantizer to produce enhanced labels, while anchoring labels derived directly from the raw input stabilize training and prevent collapse. Training is formulated as an efficient first-order bilevel optimization problem, solved end-to-end with differentiable Gumbel-softmax selection. This design eliminates the need for external label encoders, reduces memory cost, and enables iterative label refinement in an end-to-end fashion. BiRQ consistently improves over BEST-RQ while maintaining low complexity and computational efficiency. We validate our method on various datasets, including 960-hour LibriSpeech, 150-hour AMI meetings and 5,000-hour YODAS, demonstrating consistent gains over BEST-RQ.

Method: Two-phase framework: Phase 1 translates natural language personas into multidimensional psycholinguistic profiles with normalized scores. Phase 2 uses these profiles to guide generation along measurable axes of variation. Evaluated across three LLMs using three steering approaches.

Result: Schema-based approaches significantly reduce artificial persona repetition and improve coherence (silhouette scores: 0.098→0.237, topic purity: 0.773→0.957). SBS produces more concise outputs with higher topical consistency, while NPS offers greater lexical diversity. HPS achieves a balance between these extremes.

Conclusion: PILOT effectively maintains high response quality across all steering approaches, demonstrating that structured psycholinguistic profiles provide better control over LLM outputs than natural language personas alone.

Abstract: Generative AI applications commonly leverage user personas as a steering mechanism for synthetic data generation, but reliance on natural language representations forces models to make unintended inferences about which attributes to emphasize, limiting precise control over outputs. We introduce PILOT (Psychological and Linguistic Output Targeting), a two-phase framework for steering large language models with structured psycholinguistic profiles. In Phase 1, PILOT translates natural language persona descriptions into multidimensional profiles with normalized scores across linguistic and psychological dimensions. In Phase 2, these profiles guide generation along measurable axes of variation. We evaluate PILOT across three state-of-the-art LLMs (Mistral Large 2, Deepseek-R1, LLaMA 3.3 70B) using 25 synthetic personas under three conditions: Natural-language Persona Steering (NPS), Schema-Based Steering (SBS), and Hybrid Persona-Schema Steering (HPS). Results demonstrate that schema-based approaches significantly reduce artificial-sounding persona repetition while improving output coherence, with silhouette scores increasing from 0.098 to 0.237 and topic purity from 0.773 to 0.957. Our analysis reveals a fundamental trade-off: SBS produces more concise outputs with higher topical consistency, while NPS offers greater lexical diversity but reduced predictability. HPS achieves a balance between these extremes, maintaining output variety while preserving structural consistency. Expert linguistic evaluation confirms that PILOT maintains high response quality across all conditions, with no statistically significant differences between steering approaches.

Method: 26 red teamers generated 726 prompts targeting illegal activity, disinformation, and unethical behavior. 17 annotators rated 2,904 model outputs on 5-point harmfulness scale across text-only and multimodal formats.

Result: Pixtral 12B had highest harmful response rate (~62%), Claude Sonnet 3.5 most resistant (~10%). Text-only prompts slightly more effective than multimodal ones. Model type and input modality were significant predictors of harmfulness.

Conclusion: Findings underscore urgent need for robust multimodal safety benchmarks as MLLMs are deployed more widely, highlighting significant safety gaps in current models.

Abstract: Multimodal large language models (MLLMs) are increasingly used in real world applications, yet their safety under adversarial conditions remains underexplored. This study evaluates the harmlessness of four leading MLLMs (GPT-4o, Claude Sonnet 3.5, Pixtral 12B, and Qwen VL Plus) when exposed to adversarial prompts across text-only and multimodal formats. A team of 26 red teamers generated 726 prompts targeting three harm categories: illegal activity, disinformation, and unethical behaviour. These prompts were submitted to each model, and 17 annotators rated 2,904 model outputs for harmfulness using a 5-point scale. Results show significant differences in vulnerability across models and modalities. Pixtral 12B exhibited the highest rate of harmful responses (~62%), while Claude Sonnet 3.5 was the most resistant (~10%). Contrary to expectations, text-only prompts were slightly more effective at bypassing safety mechanisms than multimodal ones. Statistical analysis confirmed that both model type and input modality were significant predictors of harmfulness. These findings underscore the urgent need for robust, multimodal safety benchmarks as MLLMs are deployed more widely.

Method: Applies conformal prediction to generate prediction sets with coverage guarantees, then computes weighted averages using softmax-renormalized probabilities over the conformal sets for uncertainty-aware decoding.

Result: Consistent QWK improvements of 1-3 points across different base models. Achieved QWK scores of 84.9%(test) and 85.7%(blind test) for sentence level, and 73.3% for document level in the strict track.

Conclusion: The approach enables human reviewers to focus on a handful of plausible readability levels, effectively combining statistical guarantees with practical usability for Arabic educational assessment.

Abstract: We present a simple, model-agnostic post-processing technique for fine-grained Arabic readability classification in the BAREC 2025 Shared Task (19 ordinal levels). Our method applies conformal prediction to generate prediction sets with coverage guarantees, then computes weighted averages using softmax-renormalized probabilities over the conformal sets. This uncertainty-aware decoding improves Quadratic Weighted Kappa (QWK) by reducing high-penalty misclassifications to nearer levels. Our approach shows consistent QWK improvements of 1-3 points across different base models. In the strict track, our submission achieves QWK scores of 84.9%(test) and 85.7% (blind test) for sentence level, and 73.3% for document level. For Arabic educational assessment, this enables human reviewers to focus on a handful of plausible levels, combining statistical guarantees with practical usability.

Method: Treat optimal cache selection as a knapsack problem and employ an accumulation-based strategy to balance computational overhead and cache updates.

Result: The algorithm achieves an O(√MNT) regret bound (improving from O(MN√T)) and provides a problem-dependent bound. Experiments show approximately 12% total cost reduction on real-world data.

Conclusion: The proposed approach effectively handles query heterogeneity in LLM cache bandit problems, achieving superior theoretical guarantees and practical cost savings compared to previous methods.

Abstract: This paper revisits the LLM cache bandit problem, with a special focus on addressing the query heterogeneity for cost-effective LLM inference. Previous works often assume uniform query sizes. Heterogeneous query sizes introduce a combinatorial structure for cache selection, making the cache replacement process more computationally and statistically challenging. We treat optimal cache selection as a knapsack problem and employ an accumulation-based strategy to effectively balance computational overhead and cache updates. In theoretical analysis, we prove that the regret of our algorithm achieves an $O(\sqrt{MNT})$ bound, improving the coefficient of $\sqrt{MN}$ compared to the $O(MN\sqrt{T})$ result in Berkeley, where $N$ is the total number of queries and $M$ is the cache size. Additionally, we also provide a problem-dependent bound, which was absent in previous works. The experiment rely on real-world data show that our algorithm reduces the total cost by approximately 12%.

Method: Systematic comparison framework evaluating three aspects: systematic biases in slang perception, creativity (lexical coinages and word reuses), and informativeness for model distillation. Uses human slang from Online Slang Dictionary vs. GPT-4o and Llama-3 generated slang.

Result: Significant biases found in how LLMs perceive slang. LLMs capture creative aspects of slang but this knowledge doesn’t sufficiently align with humans for extrapolative tasks like linguistic analyses.

Conclusion: LLMs have significant knowledge about slang’s creative aspects, but this knowledge doesn’t align well enough with human usage to enable reliable extrapolative linguistic tasks.

Abstract: Slang is a commonly used type of informal language that poses a daunting challenge to NLP systems. Recent advances in large language models (LLMs), however, have made the problem more approachable. While LLM agents are becoming more widely applied to intermediary tasks such as slang detection and slang interpretation, their generalizability and reliability are heavily dependent on whether these models have captured structural knowledge about slang that align well with human attested slang usages. To answer this question, we contribute a systematic comparison between human and machine-generated slang usages. Our evaluative framework focuses on three core aspects: 1) Characteristics of the usages that reflect systematic biases in how machines perceive slang, 2) Creativity reflected by both lexical coinages and word reuses employed by the slang usages, and 3) Informativeness of the slang usages when used as gold-standard examples for model distillation. By comparing human-attested slang usages from the Online Slang Dictionary (OSD) and slang generated by GPT-4o and Llama-3, we find significant biases in how LLMs perceive slang. Our results suggest that while LLMs have captured significant knowledge about the creative aspects of slang, such knowledge does not align with humans sufficiently to enable LLMs for extrapolative tasks such as linguistic analyses.

Method: Proposes chunk-based SSL with masked prediction loss, where an acoustic encoder restores masked speech frames using unmasked frames in the same and preceding chunks. Uses finite scalar quantization (FSQ) with large codebooks, copy-and-append data augmentation, and group masked prediction to handle memory/computation costs.

Result: Experimental results on Librispeech and Must-C datasets show the method achieves very competitive results for speech-to-text tasks in both streaming and offline modes.

Conclusion: Chunk SSL provides an effective unified solution for streaming and offline speech pre-training, demonstrating strong performance on speech recognition and translation tasks while addressing the limitations of full-utterance SSL approaches.

Abstract: Low latency speech human-machine communication is becoming increasingly necessary as speech technology advances quickly in the last decade. One of the primary factors behind the advancement of speech technology is self-supervised learning. Most self-supervised learning algorithms are designed with full utterance assumption and compromises have to made if partial utterances are presented, which are common in the streaming applications. In this work, we propose a chunk based self-supervised learning (Chunk SSL) algorithm as an unified solution for both streaming and offline speech pre-training. Chunk SSL is optimized with the masked prediction loss and an acoustic encoder is encouraged to restore indices of those masked speech frames with help from unmasked frames in the same chunk and preceding chunks. A copy and append data augmentation approach is proposed to conduct efficient chunk based pre-training. Chunk SSL utilizes a finite scalar quantization (FSQ) module to discretize input speech features and our study shows a high resolution FSQ codebook, i.e., a codebook with vocabulary size up to a few millions, is beneficial to transfer knowledge from the pre-training task to the downstream tasks. A group masked prediction loss is employed during pre-training to alleviate the high memory and computation cost introduced by the large codebook. The proposed approach is examined in two speech to text tasks, i.e., speech recognition and speech translation. Experimental results on the \textsc{Librispeech} and \textsc{Must-C} datasets show that the proposed method could achieve very competitive results for speech to text tasks at both streaming and offline modes.

Method: Introduces Multilingual Data Quality and Diversity (M-DaQ) method that selects high-quality and semantically diverse multilingual IFT samples, and systematically investigates the Superficial Alignment Hypothesis in multilingual settings.

Result: Models fine-tuned with M-DaQ achieve significant performance gains over vanilla baselines with over 60% win rate across 18 languages. Human evaluations show increased cultural points in responses.

Conclusion: M-DaQ effectively addresses multilingual data scarcity and improves LLM multilinguality, with released code to support future research.

Abstract: Multilingual Instruction Fine-Tuning (IFT) is essential for enabling large language models (LLMs) to generalize effectively across diverse linguistic and cultural contexts. However, the scarcity of high-quality multilingual training data and corresponding building method remains a critical bottleneck. While data selection has shown promise in English settings, existing methods often fail to generalize across languages due to reliance on simplistic heuristics or language-specific assumptions. In this work, we introduce Multilingual Data Quality and Diversity (M-DaQ), a novel method for improving LLMs multilinguality, by selecting high-quality and semantically diverse multilingual IFT samples. We further conduct the first systematic investigation of the Superficial Alignment Hypothesis (SAH) in multilingual setting. Empirical results across 18 languages demonstrate that models fine-tuned with M-DaQ method achieve significant performance gains over vanilla baselines over 60% win rate. Human evaluations further validate these gains, highlighting the increment of cultural points in the response. We release the M-DaQ code to support future research.

Method: Proposes a DNA-inspired repair-based process that constructs ideal AI-generated sequences, iteratively repairs non-optimal tokens, and quantifies cumulative repair effort as an interpretable detection signal.

Result: Achieves state-of-the-art detection performance with strong robustness against adversarial attacks and input lengths, showing 5.55% AUROC and 2.08% F1 score improvements across multiple benchmark datasets.

Conclusion: DNA-DetectLLM provides an effective zero-shot solution for distinguishing AI-generated text, offering interpretable detection signals and robust performance in challenging detection scenarios.

Abstract: The rapid advancement of large language models (LLMs) has blurred the line between AI-generated and human-written text. This progress brings societal risks such as misinformation, authorship ambiguity, and intellectual property concerns, highlighting the urgent need for reliable AI-generated text detection methods. However, recent advances in generative language modeling have resulted in significant overlap between the feature distributions of human-written and AI-generated text, blurring classification boundaries and making accurate detection increasingly challenging. To address the above challenges, we propose a DNA-inspired perspective, leveraging a repair-based process to directly and interpretably capture the intrinsic differences between human-written and AI-generated text. Building on this perspective, we introduce DNA-DetectLLM, a zero-shot detection method for distinguishing AI-generated and human-written text. The method constructs an ideal AI-generated sequence for each input, iteratively repairs non-optimal tokens, and quantifies the cumulative repair effort as an interpretable detection signal. Empirical evaluations demonstrate that our method achieves state-of-the-art detection performance and exhibits strong robustness against various adversarial attacks and input lengths. Specifically, DNA-DetectLLM achieves relative improvements of 5.55% in AUROC and 2.08% in F1 score across multiple public benchmark datasets.

Method: Using minimal pair designs and diagnostic feature analysis across 71 tasks spanning diverse linguistic levels, the researchers conducted layer-wise and time-resolved analysis of various SLM types including self-supervised learning (S3M), automatic speech recognition (ASR), speech compression (codec), and auditory large language model encoders.

Result: The analysis reveals that all speech models encode grammatical features more robustly than conceptual features.

Conclusion: The study provides the first systematic evaluation of contextual syntactic and semantic feature encoding in speech language models, establishing that grammatical features are more strongly represented than conceptual ones across different SLM architectures.

Abstract: Transformer-based speech language models (SLMs) have significantly improved neural speech recognition and understanding. While existing research has examined how well SLMs encode shallow acoustic and phonetic features, the extent to which SLMs encode nuanced syntactic and conceptual features remains unclear. By drawing parallels with linguistic competence assessments for large language models, this study is the first to systematically evaluate the presence of contextual syntactic and semantic features across SLMs for self-supervised learning (S3M), automatic speech recognition (ASR), speech compression (codec), and as the encoder for auditory large language models (AudioLLMs). Through minimal pair designs and diagnostic feature analysis across 71 tasks spanning diverse linguistic levels, our layer-wise and time-resolved analysis uncovers that 1) all speech encode grammatical features more robustly than conceptual ones.

Method: Climb introduces a cross-lingual interaction-aware language ratio to quantify each language’s effective allocation by capturing inter-language dependencies, then uses a two-step optimization procedure: first equalizing marginal benefits across languages, then maximizing the magnitude of resulting language allocation vectors.

Result: Extensive experiments show Climb accurately measures cross-lingual interactions across various multilingual settings. LLMs trained with Climb-derived proportions consistently achieve state-of-the-art multilingual performance, even matching open-sourced LLMs trained with more tokens.

Conclusion: Climb provides a systematic framework for optimizing multilingual data allocation that significantly simplifies the complex multilingual optimization problem and delivers superior multilingual performance in language models.

Abstract: Large language models (LLMs) have become integral to a wide range of applications worldwide, driving an unprecedented global demand for effective multilingual capabilities. Central to achieving robust multilingual performance is the strategic allocation of language proportions within training corpora. However, determining optimal language ratios is highly challenging due to intricate cross-lingual interactions and sensitivity to dataset scale. This paper introduces Climb (Cross-Lingual Interaction-aware Multilingual Balancing), a novel framework designed to systematically optimize multilingual data allocation. At its core, Climb introduces a cross-lingual interaction-aware language ratio, explicitly quantifying each language’s effective allocation by capturing inter-language dependencies. Leveraging this ratio, Climb proposes a principled two-step optimization procedure–first equalizing marginal benefits across languages, then maximizing the magnitude of the resulting language allocation vectors–significantly simplifying the inherently complex multilingual optimization problem. Extensive experiments confirm that Climb can accurately measure cross-lingual interactions across various multilingual settings. LLMs trained with Climb-derived proportions consistently achieve state-of-the-art multilingual performance, even achieving competitive performance with open-sourced LLMs trained with more tokens.

Method: Fuses Whisper’s hidden decoder states with LLM states through cross-modal attention in continuous text representation spaces, supporting both offline and streaming modes.

Result: Created VoxKrikri (first Greek speech LLM) and achieved ~20% relative improvement in Greek ASR benchmarks, demonstrating effective cross-modal alignment.

Conclusion: Continuous space fusion is a promising approach for multilingual and low-resource speech LLMs, providing state-of-the-art performance while effectively bridging audio and text modalities.

Abstract: We present a multimodal fusion framework that bridges pre-trained decoder-based large language models (LLM) and acoustic encoder-decoder architectures such as Whisper, with the aim of building speech-enabled LLMs. Instead of directly using audio embeddings, we explore an intermediate audio-conditioned text space as a more effective mechanism for alignment. Our method operates fully in continuous text representation spaces, fusing Whisper’s hidden decoder states with those of an LLM through cross-modal attention, and supports both offline and streaming modes. We introduce \textit{VoxKrikri}, the first Greek speech LLM, and show through analysis that our approach effectively aligns representations across modalities. These results highlight continuous space fusion as a promising path for multilingual and low-resource speech LLMs, while achieving state-of-the-art results for Automatic Speech Recognition in Greek, providing an average $\sim20%$ relative improvement across benchmarks.

Method: The paper analyzes two key properties of language: (1) compact representations that facilitate reasoning about abstract concepts, and (2) language as the collective output of cultural evolution containing accumulated abstractions.

Result: Language serves as a powerful tool for developing domain-general abilities by providing compressed models of the world that capture conceptual and causal structures supporting human thought.

Conclusion: Language exposure allows learning systems (biological or artificial) to reverse-engineer human-like conceptual structures, making language fundamental to both human intelligence and the development of more general AI systems.

Abstract: We use language to communicate our thoughts. But is language merely the expression of thoughts, which are themselves produced by other, nonlinguistic parts of our minds? Or does language play a more transformative role in human cognition, allowing us to have thoughts that we otherwise could (or would) not have? Recent developments in artificial intelligence (AI) and cognitive science have reinvigorated this old question. We argue that language may hold the key to the emergence of both more general AI systems and central aspects of human intelligence. We highlight two related properties of language that make it such a powerful tool for developing domain–general abilities. First, language offers compact representations that make it easier to represent and reason about many abstract concepts (e.g., exact numerosity). Second, these compressed representations are the iterated output of collective minds. In learning a language, we learn a treasure trove of culturally evolved abstractions. Taken together, these properties mean that a sufficiently powerful learning system exposed to language–whether biological or artificial–learns a compressed model of the world, reverse engineering many of the conceptual and causal structures that support human (and human-like) thought.

Method: Fine-tuning LMMs using the Speechocean762 dataset and a private corpus, comparing performance against zero-shot settings and existing public/commercial systems across multiple granularities (phoneme, word, sentence levels).

Result: Fine-tuning significantly outperforms zero-shot settings and achieves competitive results on single-granularity tasks. The model performs well at word and sentence levels but struggles with phoneme-level assessment. PCC reaches 0.9 while SCC remains around 0.6, suggesting SCC better reflects ordinal consistency.

Conclusion: LMMs show promise for APA but have limitations in fine-grained assessment, particularly at the phoneme level. Future work should focus on fine-grained modeling and rank-aware evaluation methods.

Abstract: Automatic Pronunciation Assessment (APA) is critical for Computer-Assisted Language Learning (CALL), requiring evaluation across multiple granularities and aspects. Large Multimodal Models (LMMs) present new opportunities for APA, but their effectiveness in fine-grained assessment remains uncertain. This work investigates fine-tuning LMMs for APA using the Speechocean762 dataset and a private corpus. Fine-tuning significantly outperforms zero-shot settings and achieves competitive results on single-granularity tasks compared to public and commercial systems. The model performs well at word and sentence levels, while phoneme-level assessment remains challenging. We also observe that the Pearson Correlation Coefficient (PCC) reaches 0.9, whereas Spearman’s rank Correlation Coefficient (SCC) remains around 0.6, suggesting that SCC better reflects ordinal consistency. These findings highlight both the promise and limitations of LMMs for APA and point to future work on fine-grained modeling and rank-aware evaluation.

Method: Uses BISAC book classification system for hierarchical topic organization, multi-agent debate with multiple LLMs to generate diverse topics, and lightweight BM25 retrieval to create 128K-token training samples.

Result: Achieves competitive long-context performance on HELMET and Ruler benchmarks, and can integrate with other long-dependency enhancement methods.

Conclusion: LiteLong makes high-quality long-context data synthesis more accessible by reducing computational and data engineering costs, facilitating further research in long-context language training.

Abstract: High-quality long-context data is essential for training large language models (LLMs) capable of processing extensive documents, yet existing synthesis approaches using relevance-based aggregation face challenges of computational efficiency. We present LiteLong, a resource-efficient method for synthesizing long-context data through structured topic organization and multi-agent debate. Our approach leverages the BISAC book classification system to provide a comprehensive hierarchical topic organization, and then employs a debate mechanism with multiple LLMs to generate diverse, high-quality topics within this structure. For each topic, we use lightweight BM25 retrieval to obtain relevant documents and concatenate them into 128K-token training samples. Experiments on HELMET and Ruler benchmarks demonstrate that LiteLong achieves competitive long-context performance and can seamlessly integrate with other long-dependency enhancement methods. LiteLong makes high-quality long-context data synthesis more accessible by reducing both computational and data engineering costs, facilitating further research in long-context language training.

Method: Proposes R2U with direct optimization to maximize probability of generating correct answers through process supervision. Uses efficient distillation pipeline with scaled supervision from LLMs to help smaller rewriter models generalize better.

Result: Empirical evaluation across multiple open-domain question-answering benchmarks shows consistent improvements over strong bridging baselines.

Conclusion: Direct optimization for generative utility through process supervision effectively bridges the gap between retrieval relevance and generation performance in RAG systems.

Abstract: Retrieval-Augmented Generation systems often suffer from a gap between optimizing retrieval relevance and generative utility: retrieved documents may be topically relevant but still lack the content needed for effective reasoning during generation. While existing “bridge” modules attempt to rewrite the retrieved text for better generation, we show how they fail to capture true document utility. In this work, we propose R2U, with a key distinction of directly optimizing to maximize the probability of generating a correct answer through process supervision. As such direct observation is expensive, we also propose approximating an efficient distillation pipeline by scaling the supervision from LLMs, which helps the smaller rewriter model generalize better. We evaluate our method across multiple open-domain question-answering benchmarks. The empirical results demonstrate consistent improvements over strong bridging baselines.

Method: Proposed DivLogicEval benchmark consisting of natural sentences composed of diverse statements in counterintuitive ways, and introduced a new evaluation metric to mitigate bias and randomness in LLM evaluations.

Result: Experiments demonstrated that logical reasoning is genuinely required to answer DivLogicEval questions, and the benchmark effectively compared the logical reasoning performance of different popular LLMs.

Conclusion: DivLogicEval provides a more reliable and faithful evaluation framework for assessing LLMs’ logical reasoning abilities, addressing limitations of existing benchmarks through diverse language composition and improved evaluation metrics.

Abstract: Logic reasoning in natural language has been recognized as an important measure of human intelligence for Large Language Models (LLMs). Popular benchmarks may entangle multiple reasoning skills and thus provide unfaithful evaluations on the logic reasoning skill. Meanwhile, existing logic reasoning benchmarks are limited in language diversity and their distributions are deviated from the distribution of an ideal logic reasoning benchmark, which may lead to biased evaluation results. This paper thereby proposes a new classical logic benchmark DivLogicEval, consisting of natural sentences composed of diverse statements in a counterintuitive way. To ensure a more reliable evaluation, we also introduce a new evaluation metric that mitigates the influence of bias and randomness inherent in LLMs. Through experiments, we demonstrate the extent to which logical reasoning is required to answer the questions in DivLogicEval and compare the performance of different popular LLMs in conducting logical reasoning.

Method: The paper introduces SciEvent as a multi-stage event extraction pipeline: (1) segmenting abstracts into core scientific activities (Background, Method, Result, Conclusion), and (2) extracting corresponding triggers and arguments. The benchmark includes 500 abstracts across five research domains with manual annotations.

Result: Experiments with fine-tuned EE models, LLMs, and human annotators reveal a performance gap, with current models struggling particularly in domains like sociology and humanities.

Conclusion: SciEvent serves as a challenging benchmark and represents a step toward generalizable, multi-domain scientific information extraction that can better handle interdisciplinary research contexts.

Abstract: Scientific information extraction (SciIE) has primarily relied on entity-relation extraction in narrow domains, limiting its applicability to interdisciplinary research and struggling to capture the necessary context of scientific information, often resulting in fragmented or conflicting statements. In this paper, we introduce SciEvent, a novel multi-domain benchmark of scientific abstracts annotated via a unified event extraction (EE) schema designed to enable structured and context-aware understanding of scientific content. It includes 500 abstracts across five research domains, with manual annotations of event segments, triggers, and fine-grained arguments. We define SciIE as a multi-stage EE pipeline: (1) segmenting abstracts into core scientific activities–Background, Method, Result, and Conclusion; and (2) extracting the corresponding triggers and arguments. Experiments with fine-tuned EE models, large language models (LLMs), and human annotators reveal a performance gap, with current models struggling in domains such as sociology and humanities. SciEvent serves as a challenging benchmark and a step toward generalizable, multi-domain SciIE.

Method: The approach leverages knowledge graphs to represent LLM knowledge, defines CU as removing target nodes and edges, and uses prompting to generate knowledge triplets and explanatory sentences about forgetting targets for precise unlearning.

Result: Experiments on real-world and synthetic datasets show the method effectively achieves concept-level unlearning while preserving unrelated knowledge.

Conclusion: The proposed Concept Unlearning framework enables more intuitive and comprehensive concept removal by aligning with LLM’s internal knowledge representations, addressing limitations of existing sentence-level unlearning approaches.

Abstract: Machine Unlearning (MU) has recently attracted considerable attention as a solution to privacy and copyright issues in large language models (LLMs). Existing MU methods aim to remove specific target sentences from an LLM while minimizing damage to unrelated knowledge. However, these approaches require explicit target sentences and do not support removing broader concepts, such as persons or events. To address this limitation, we introduce Concept Unlearning (CU) as a new requirement for LLM unlearning. We leverage knowledge graphs to represent the LLM’s internal knowledge and define CU as removing the forgetting target nodes and associated edges. This graph-based formulation enables a more intuitive unlearning and facilitates the design of more effective methods. We propose a novel method that prompts the LLM to generate knowledge triplets and explanatory sentences about the forgetting target and applies the unlearning process to these representations. Our approach enables more precise and comprehensive concept removal by aligning the unlearning process with the LLM’s internal knowledge representations. Experiments on real-world and synthetic datasets demonstrate that our method effectively achieves concept-level unlearning while preserving unrelated knowledge.

Method: Proposes an unlearning objective that modifies target entity activations away from known entities and toward unknown entities in sparse autoencoder latent space, shifting recognition from ‘known’ to ‘unknown’.

Result: Effectively aligns internal activations of forgotten targets, reduces model recall of target knowledge in QA tasks without significant damage to non-target knowledge, outperforms suppression-based approaches.

Conclusion: Direct intervention in internal activations achieves genuine forgetting while avoiding over-suppression and model collapse, representing a more effective approach to LLM unlearning.

Abstract: As large language models (LLMs) are increasingly deployed across various applications, privacy and copyright concerns have heightened the need for more effective LLM unlearning techniques. Many existing unlearning methods aim to suppress undesirable outputs through additional training (e.g., gradient ascent), which reduces the probability of generating such outputs. While such suppression-based approaches can control model outputs, they may not eliminate the underlying knowledge embedded in the model’s internal activations; muting a response is not the same as forgetting it. Moreover, such suppression-based methods often suffer from model collapse. To address these issues, we propose a novel unlearning method that directly intervenes in the model’s internal activations. In our formulation, forgetting is defined as a state in which the activation of a forgotten target is indistinguishable from that of unknown'' entities. Our method introduces an unlearning objective that modifies the activation of the target entity away from those of known entities and toward those of unknown entities in a sparse autoencoder latent space. By aligning the target's internal activation with those of unknown entities, we shift the model's recognition of the target entity from known’’ to ``unknown’’, achieving genuine forgetting while avoiding over-suppression and model collapse. Empirically, we show that our method effectively aligns the internal activations of the forgotten target, a result that the suppression-based approaches do not reliably achieve. Additionally, our method effectively reduces the model’s recall of target knowledge in question-answering tasks without significant damage to the non-target knowledge.

Method: Systematically evaluated six multilingual LLMs (0.5B-9B parameters) on MedEV dataset using zero-shot, few-shot, and dictionary-augmented prompting with Meddict medical lexicon.

Result: Larger LLMs achieve strong zero-shot results, few-shot prompting offers only marginal improvements, but terminology-aware cues and embedding-based example retrieval consistently improve domain-specific translation.

Conclusion: Multilingual LLMs show promise for medical En-Vi MT but have current limitations, with model scale being the primary performance driver.

Abstract: Medical English-Vietnamese machine translation (En-Vi MT) is essential for healthcare access and communication in Vietnam, yet Vietnamese remains a low-resource and under-studied language. We systematically evaluate prompting strategies for six multilingual LLMs (0.5B-9B parameters) on the MedEV dataset, comparing zero-shot, few-shot, and dictionary-augmented prompting with Meddict, an English-Vietnamese medical lexicon. Results show that model scale is the primary driver of performance: larger LLMs achieve strong zero-shot results, while few-shot prompting yields only marginal improvements. In contrast, terminology-aware cues and embedding-based example retrieval consistently improve domain-specific translation. These findings underscore both the promise and the current limitations of multilingual LLMs for medical En-Vi MT.

Method: Train a student model to generate stories, which a teacher model rates on readability, narrative coherence, and creativity. Vary pretraining amounts before implementing the feedback loop to assess impact.

Result: High-level feedback is highly data efficient: with just 1M words in interactive learning, storytelling skills improve as much as with 410M words of next-word prediction.

Conclusion: Interactive learning with cognitive feedback can dramatically reduce data requirements for language model training while achieving comparable performance gains.

Abstract: Children efficiently acquire language not just by listening, but by interacting with others in their social environment. Conversely, large language models are typically trained with next-word prediction on massive amounts of text. Motivated by this contrast, we investigate whether language models can be trained with less data by learning not only from next-word prediction but also from high-level, cognitively inspired feedback. We train a student model to generate stories, which a teacher model rates on readability, narrative coherence, and creativity. By varying the amount of pretraining before the feedback loop, we assess the impact of this interactive learning on formal and functional linguistic competence. We find that the high-level feedback is highly data efficient: With just 1 M words of input in interactive learning, storytelling skills can improve as much as with 410 M words of next-word prediction.

Method: The study adapts frequency-based representations from cognitive science research (which reduce human statistical reasoning biases) to create REFER - a frequency-framed prompting framework. This approach makes reference classes explicit and reduces cognitive load for LLMs, similar to how it improves human reasoning.

Result: REFER enhances fairness in language models when summarizing opinions. The effect is particularly pronounced in larger language models and when using stronger reasoning instructions.

Conclusion: Frequency-framed prompting (REFER) effectively improves fairness in LLM opinion summarization by applying cognitive science principles to reduce systematic biases, offering a practical alternative to previous methods that required hyperparameter tuning or distributional information.

Abstract: Individuals express diverse opinions, a fair summary should represent these viewpoints comprehensively. Previous research on fairness in opinion summarisation using large language models (LLMs) relied on hyperparameter tuning or providing ground truth distributional information in prompts. However, these methods face practical limitations: end-users rarely modify default model parameters, and accurate distributional information is often unavailable. Building upon cognitive science research demonstrating that frequency-based representations reduce systematic biases in human statistical reasoning by making reference classes explicit and reducing cognitive load, this study investigates whether frequency framed prompting (REFER) can similarly enhance fairness in LLM opinion summarisation. Through systematic experimentation with different prompting frameworks, we adapted techniques known to improve human reasoning to elicit more effective information processing in language models compared to abstract probabilistic representations.Our results demonstrate that REFER enhances fairness in language models when summarising opinions. This effect is particularly pronounced in larger language models and using stronger reasoning instructions.

Method: Using Quantitative Argumentation Debate (QuAD) semantics on dialogue-formatted debates from NoDE datasets, testing several LLMs with advanced instruction strategies including Chain-of-Thought and In-Context Learning to rank arguments.

Result: LLMs show moderate alignment with QuAD rankings, but performance degrades with longer inputs or disrupted discourse flow. Advanced prompting helps mitigate biases related to argument length and position.

Conclusion: The findings highlight both the promise and limitations of LLMs in modeling formal argumentation semantics and motivate future work on graph-aware reasoning.

Abstract: Large Language Models (LLMs) excel at linear reasoning tasks but remain underexplored on non-linear structures such as those found in natural debates, which are best expressed as argument graphs. We evaluate whether LLMs can approximate structured reasoning from Computational Argumentation Theory (CAT). Specifically, we use Quantitative Argumentation Debate (QuAD) semantics, which assigns acceptability scores to arguments based on their attack and support relations. Given only dialogue-formatted debates from two NoDE datasets, models are prompted to rank arguments without access to the underlying graph. We test several LLMs under advanced instruction strategies, including Chain-of-Thought and In-Context Learning. While models show moderate alignment with QuAD rankings, performance degrades with longer inputs or disrupted discourse flow. Advanced prompting helps mitigate these effects by reducing biases related to argument length and position. Our findings highlight both the promise and limitations of LLMs in modeling formal argumentation semantics and motivate future work on graph-aware reasoning.

Method: UniGist uses a chunk-free training strategy with special compression tokens (gists) that replace raw tokens in a fine-grained manner. It employs an efficient kernel with gist shift trick for optimized GPU training and supports flexible inference by allowing actual removal of compressed tokens.

Result: Experiments across multiple long-context tasks show UniGist significantly improves compression quality, with particularly strong performance in detail-recalling tasks and long-range dependency modeling.

Conclusion: UniGist provides an effective solution for long-context compression that preserves important contextual information while achieving real-time memory savings during inference.

Abstract: Large language models are increasingly capable of handling long-context inputs, but the memory overhead of key-value (KV) cache remains a major bottleneck for general-purpose deployment. While various compression strategies have been explored, sequence-level compression, which drops the full KV caches for certain tokens, is particularly challenging as it can lead to the loss of important contextual information. To address this, we introduce UniGist, a sequence-level long-context compression framework that efficiently preserves context information by replacing raw tokens with special compression tokens (gists) in a fine-grained manner. We adopt a chunk-free training strategy and design an efficient kernel with a gist shift trick, enabling optimized GPU training. Our scheme also supports flexible inference by allowing the actual removal of compressed tokens, resulting in real-time memory savings. Experiments across multiple long-context tasks demonstrate that UniGist significantly improves compression quality, with especially strong performance in detail-recalling tasks and long-range dependency modeling.

Method: Complete end-to-end pipeline from web scraping to text alignment using Graph-Aided Paragraph Alignment (GAPA) algorithm for paragraph-level alignment, with both single-machine and distributed computing options.

Result: Corpus contains over 713 million English tokens, more than doubling prior work’s scale, representing the largest publicly available parallel corpus of human-translated, non-AI-generated content.

Conclusion: The proposed solution successfully addresses reproducibility and scalability issues in multilingual dataset creation, providing a valuable resource for machine translation research under MIT License.

Abstract: The quality and accessibility of multilingual datasets are crucial for advancing machine translation. However, previous corpora built from United Nations documents have suffered from issues such as opaque process, difficulty of reproduction, and limited scale. To address these challenges, we introduce a complete end-to-end solution, from data acquisition via web scraping to text alignment. The entire process is fully reproducible, with a minimalist single-machine example and optional distributed computing steps for scalability. At its core, we propose a new Graph-Aided Paragraph Alignment (GAPA) algorithm for efficient and flexible paragraph-level alignment. The resulting corpus contains over 713 million English tokens, more than doubling the scale of prior work. To the best of our knowledge, this represents the largest publicly available parallel corpus composed entirely of human-translated, non-AI-generated content. Our code and corpus are accessible under the MIT License.

Method: The LESS framework refines pseudo-labeled text from ASR/AST using LLMs and incorporates a data filtering strategy to improve quality.

Result: Achieved 3.8% absolute WER reduction on WenetSpeech (Mandarin ASR) and BLEU score increases of 0.8 and 0.7, reaching 34.0 on Callhome and 64.7 on Fisher testsets (Spanish-to-English AST).

Conclusion: LESS demonstrates effectiveness across diverse languages, tasks, and domains, with the recipe released as open source to facilitate further research.

Abstract: Although state-of-the-art Speech Foundation Models can produce high-quality text pseudo-labels, applying Semi-Supervised Learning (SSL) for in-the-wild real-world data remains challenging due to its richer and more complex acoustics compared to curated datasets. To address the challenges, we introduce LESS (Large Language Model Enhanced Semi-supervised Learning), a versatile framework that uses Large Language Models (LLMs) to correct pseudo-labels generated on in-the-wild data. In the LESS framework, pseudo-labeled text from Automatic Speech Recognition (ASR) or Automatic Speech Translation (AST) of the unsupervised data is refined by an LLM, and further improved by a data filtering strategy. Across Mandarin ASR and Spanish-to-English AST evaluations, LESS delivers consistent gains, with an absolute Word Error Rate reduction of 3.8% on WenetSpeech, and BLEU score increase of 0.8 and 0.7, achieving 34.0 on Callhome and 64.7 on Fisher testsets respectively. These results highlight LESS’s effectiveness across diverse languages, tasks, and domains. We have released the recipe as open source to facilitate further research in this area.

Method: RAVE (Retrieval and Scoring Aware Verifiable Claim Detection) integrates evidence retrieval with structured signals of relevance and source credibility.

Result: Experiments on CT22-test and PoliClaim-test datasets show RAVE consistently outperforms baselines in both accuracy and F1 scores.

Conclusion: The RAVE framework provides an effective approach for verifiable claim detection by leveraging evidence retrieval and credibility assessment.

Abstract: The rapid spread of misinformation on social media underscores the need for scalable fact-checking tools. A key step is claim detection, which identifies statements that can be objectively verified. Prior approaches often rely on linguistic cues or claim check-worthiness, but these struggle with vague political discourse and diverse formats such as tweets. We present RAVE (Retrieval and Scoring Aware Verifiable Claim Detection), a framework that combines evidence retrieval with structured signals of relevance and source credibility. Experiments on CT22-test and PoliClaim-test show that RAVE consistently outperforms text-only and retrieval-based baselines in both accuracy and F1.

Method: Train a reward model to rank generated responses for questions across different languages, enabling cross-lingual sampling and evaluation.

Result: Cross-lingual reward modeling substantially improves mathematical reasoning performance compared to single-language approaches, benefiting even high-resource languages like English, especially under low sampling budgets.

Conclusion: Multilingual reasoning can be improved by leveraging the complementary strengths of diverse languages through cross-lingual approaches.

Abstract: While the reasoning abilities of large language models (LLMs) continue to advance, it remains unclear how such ability varies across languages in multilingual LLMs and whether different languages produce reasoning paths that complement each other. To investigate this question, we train a reward model to rank generated responses for a given question across languages. Our results show that our cross-lingual reward model substantially improves mathematical reasoning performance compared to using reward modeling within a single language, benefiting even high-resource languages. While English often exhibits the highest performance in multilingual models, we find that cross-lingual sampling particularly benefits English under low sampling budgets. Our findings reveal new opportunities to improve multilingual reasoning by leveraging the complementary strengths of diverse languages.

Method: Used global representational comparisons and targeted clustering analyses to probe phonetic vs. semantic discriminability in model representations of speech-based and text-based language encoders.

Result: Visual grounding increases alignment between spoken and written language representations, but this is driven by enhanced word identity encoding rather than meaning. Speech representations remain phonetically dominated, and visual grounding doesn’t improve semantic discriminability in text-based representations.

Conclusion: The findings can inform development of more efficient methods to enrich speech-based models with visually-informed semantics, highlighting the different effects of visual grounding on speech vs. text models.

Abstract: How does visual information included in training affect language processing in audio- and text-based deep learning models? We explore how such visual grounding affects model-internal representations of words, and find substantially different effects in speech- vs. text-based language encoders. Firstly, global representational comparisons reveal that visual grounding increases alignment between representations of spoken and written language, but this effect seems mainly driven by enhanced encoding of word identity rather than meaning. We then apply targeted clustering analyses to probe for phonetic vs. semantic discriminability in model representations. Speech-based representations remain phonetically dominated with visual grounding, but in contrast to text-based representations, visual grounding does not improve semantic discriminability. Our findings could usefully inform the development of more efficient methods to enrich speech-based models with visually-informed semantics.

Method: Created a benchmark with 5 difficulty levels and 1,412 image-associated multiple-choice questions spanning 11 high-school physics subjects. Evaluated 20 MLLMs using a dual framework analyzing both final answer accuracy and chain-of-thought reasoning integrity. Systematically studied difficulty level and visual information impact by comparing model performance across different input modes.

Result: The benchmark provides fine-grained evaluation of MLLMs’ physics reasoning capabilities, revealing gaps in current models’ performance. The study offers insights into how visual information and difficulty levels affect multimodal reasoning processes.

Conclusion: Multi-Physics serves as a valuable resource for the community and provides a robust methodology for analyzing multimodal reasoning in MLLMs, with the dataset and code being open-sourced for broader use.

Abstract: While multimodal LLMs (MLLMs) demonstrate remarkable reasoning progress, their application in specialized scientific domains like physics reveals significant gaps in current evaluation benchmarks. Specifically, existing benchmarks often lack fine-grained subject coverage, neglect the step-by-step reasoning process, and are predominantly English-centric, failing to systematically evaluate the role of visual information. Therefore, we introduce \textbf {Multi-Physics} for Chinese physics reasoning, a comprehensive benchmark that includes 5 difficulty levels, featuring 1,412 image-associated, multiple-choice questions spanning 11 high-school physics subjects. We employ a dual evaluation framework to evaluate 20 different MLLMs, analyzing both final answer accuracy and the step-by-step integrity of their chain-of-thought. Furthermore, we systematically study the impact of difficulty level and visual information by comparing the model performance before and after changing the input mode. Our work provides not only a fine-grained resource for the community but also offers a robust methodology for dissecting the multimodal reasoning process of state-of-the-art MLLMs, and our dataset and code have been open-sourced: https://github.com/luozhongze/Multi-Physics.

Method: Extract a task-aware steering vector from KL divergence gradient between warm-started and pre-trained models, then use this vector to guide decoding process toward task distribution.

Result: Improves multiple-choice accuracy by up to 5 points and open-ended truthfulness by 2 points across three tasks and nine benchmarks, with similar gains on commonsense datasets.

Conclusion: SVD provides a theoretically grounded, lightweight approach for stronger task adaptation that is compatible with existing PEFT methods and requires no additional trainable parameters.

Abstract: Adapting billion-parameter language models to a downstream task is still costly, even with parameter-efficient fine-tuning (PEFT). We re-cast task adaptation as output-distribution alignment: the objective is to steer the output distribution toward the task distribution directly during decoding rather than indirectly through weight updates. Building on this view, we introduce Steering Vector Decoding (SVD), a lightweight, PEFT-compatible, and theoretically grounded method. We start with a short warm-start fine-tune and extract a task-aware steering vector from the Kullback-Leibler (KL) divergence gradient between the output distribution of the warm-started and pre-trained models. This steering vector is then used to guide the decoding process to steer the model’s output distribution towards the task distribution. We theoretically prove that SVD is first-order equivalent to the gradient step of full fine-tuning and derive a globally optimal solution for the strength of the steering vector. Across three tasks and nine benchmarks, SVD paired with four standard PEFT methods improves multiple-choice accuracy by up to 5 points and open-ended truthfulness by 2 points, with similar gains (1-2 points) on commonsense datasets without adding trainable parameters beyond the PEFT adapter. SVD thus offers a lightweight, theoretically grounded path to stronger task adaptation for large language models.

Method: The authors analyze state-of-the-art misinformation detection systems through the lens of human psychology and behavior, exploring the interplay between traditional fact-checking and psychological concepts like cognitive biases, social dynamics, and emotional responses.

Result: The analysis reveals critical limitations in current misinformation detection methods and identifies opportunities for improvement by incorporating human psychological factors.

Conclusion: Future research should develop neuro-behavioral models that integrate technological factors with human cognition and social influence, creating more robust and adaptive frameworks to effectively detect and mitigate misinformation’s societal harms.

Abstract: Misinformation remains one of the most significant issues in the digital age. While automated fact-checking has emerged as a viable solution, most current systems are limited to evaluating factual accuracy. However, the detrimental effect of misinformation transcends simple falsehoods; it takes advantage of how individuals perceive, interpret, and emotionally react to information. This underscores the need to move beyond factuality and adopt more human-centered detection frameworks. In this survey, we explore the evolving interplay between traditional fact-checking approaches and psychological concepts such as cognitive biases, social dynamics, and emotional responses. By analyzing state-of-the-art misinformation detection systems through the lens of human psychology and behavior, we reveal critical limitations of current methods and identify opportunities for improvement. Additionally, we outline future research directions aimed at creating more robust and adaptive frameworks, such as neuro-behavioural models that integrate technological factors with the complexities of human cognition and social influence. These approaches offer promising pathways to more effectively detect and mitigate the societal harms of misinformation.

Method: FRAME extracts and scores salient facts, organizes them thematically, and enriches an outline into an abstractive summary. SCOPE is a reason-out-loud protocol where models answer nine questions before content selection for personalization.

Result: FRAME reduces hallucination and omission by 2 out of 5 points on QMSum and FAME datasets. P-MESA achieves >=89% balanced accuracy against human annotations and strong correlation with human severity ratings (r>=0.70). SCOPE improves knowledge fit and goal alignment over prompt-only baselines.

Conclusion: The findings advocate for rethinking summarization approaches to improve control, faithfulness, and personalization through semantic enrichment frameworks like FRAME.

Abstract: Meeting summarization with large language models (LLMs) remains error-prone, often producing outputs with hallucinations, omissions, and irrelevancies. We present FRAME, a modular pipeline that reframes summarization as a semantic enrichment task. FRAME extracts and scores salient facts, organizes them thematically, and uses these to enrich an outline into an abstractive summary. To personalize summaries, we introduce SCOPE, a reason-out-loud protocol that has the model build a reasoning trace by answering nine questions before content selection. For evaluation, we propose P-MESA, a multi-dimensional, reference-free evaluation framework to assess if a summary fits a target reader. P-MESA reliably identifies error instances, achieving >= 89% balanced accuracy against human annotations and strongly aligns with human severity ratings (r >= 0.70). On QMSum and FAME, FRAME reduces hallucination and omission by 2 out of 5 points (measured with MESA), while SCOPE improves knowledge fit and goal alignment over prompt-only baselines. Our findings advocate for rethinking summarization to improve control, faithfulness, and personalization.

Method: Fine-tuned two open-source large language models (Llama-3 8B and Qwen-2.5 3B) on three diverse corpora (ASAP, TOEFL11, Cambridge-FCE) and applied conformal prediction at a 90% risk level to provide set-valued outputs with formal coverage guarantees.

Result: The calibrated models consistently met the 90% coverage target while maintaining compact prediction sets, demonstrating that mid-sized open-source LLMs can support teacher-in-the-loop AES with reliable uncertainty quantification.

Conclusion: Conformal prediction combined with uncertainty-aware accuracy (UAcc) enables reliable AES systems, and future work should focus on scaling and broader user studies to validate real-world applicability.

Abstract: Automated Essay Scoring (AES) systems now reach near human agreement on some public benchmarks, yet real-world adoption, especially in high-stakes examinations, remains limited. A principal obstacle is that most models output a single score without any accompanying measure of confidence or explanation. We address this gap with conformal prediction, a distribution-free wrapper that equips any classifier with set-valued outputs and formal coverage guarantees. Two open-source large language models (Llama-3 8B and Qwen-2.5 3B) are fine-tuned on three diverse corpora (ASAP, TOEFL11, Cambridge-FCE) and calibrated at a 90 percent risk level. Reliability is assessed with UAcc, an uncertainty-aware accuracy that rewards models for being both correct and concise. To our knowledge, this is the first work to combine conformal prediction and UAcc for essay scoring. The calibrated models consistently meet the coverage target while keeping prediction sets compact, indicating that open-source, mid-sized LLMs can already support teacher-in-the-loop AES; we discuss scaling and broader user studies as future work.

Method: The authors propose localmax dynamics with an alignment-sensitivity parameter, analyze its convergence properties, introduce quiescent sets to capture invariant behavior, and adapt Lyapunov-based methods from opinion dynamics.

Result: The model converges to a convex polytope but cannot be fully described by maximal alignment sets alone. Quiescent sets are essential for understanding asymptotic behavior, and the system does not exhibit finite-time convergence. Results are provided for various parameter regimes.

Conclusion: Localmax dynamics offers a flexible attention framework with interesting mathematical properties, though Lyapunov methods have limitations in asymmetric settings, suggesting directions for future research.

Abstract: We introduce a new discrete-time attention model, termed the localmax dynamics, which interpolates between the classic softmax dynamics and the hardmax dynamics, where only the tokens that maximize the influence toward a given token have a positive weight. As in hardmax, uniform weights are determined by a parameter controlling neighbor influence, but the key extension lies in relaxing neighborhood interactions through an alignment-sensitivity parameter, which allows controlled deviations from pure hardmax behavior. As we prove, while the convex hull of the token states still converges to a convex polytope, its structure can no longer be fully described by a maximal alignment set, prompting the introduction of quiescent sets to capture the invariant behavior of tokens near vertices. We show that these sets play a key role in understanding the asymptotic behavior of the system, even under time-varying alignment sensitivity parameters. We further show that localmax dynamics does not exhibit finite-time convergence and provide results for vanishing, nonzero, time-varying alignment-sensitivity parameters, recovering the limiting behavior of hardmax as a by-product. Finally, we adapt Lyapunov-based methods from classical opinion dynamics, highlighting their limitations in the asymmetric setting of localmax interactions and outlining directions for future research.

Method: The authors develop a novel approach for selecting which bias term to fine-tune, forming the foundation of BEFT. They extensively evaluate this method across diverse LLMs (110M to 6.7B parameters) including both encoder-only and decoder-only architectures.

Result: The proposed bias-efficient approach demonstrates superior performance compared to other bias-selection methods on various downstream tasks including classification, multiple-choice, and generation tasks.

Conclusion: BEFT provides an effective and superior method for bias-term selection in parameter-efficient fine-tuning, offering unprecedented parameter efficiency while maintaining competitive performance across diverse language tasks and model architectures.

Abstract: Fine-tuning all-bias-terms stands out among various parameter-efficient fine-tuning (PEFT) techniques, owing to its out-of-the-box usability and competitive performance, especially in low-data regimes. Bias-only fine-tuning has the potential for unprecedented parameter efficiency. However, the link between fine-tuning different bias terms (i.e., bias terms in the query, key, or value projections) and downstream performance remains unclear. The existing approaches, e.g., based on the magnitude of bias change or empirical Fisher information, provide limited guidance for selecting the particular bias term for effective fine-tuning. In this paper, we propose an approach for selecting the bias term to be fine-tuned, forming the foundation of our bias-efficient fine-tuning (BEFT). We extensively evaluate our bias-efficient approach against other bias-selection approaches, across a wide range of large language models (LLMs) spanning encoder-only and decoder-only architectures from 110M to 6.7B parameters. Our results demonstrate the effectiveness and superiority of our bias-efficient approach on diverse downstream tasks, including classification, multiple-choice, and generation tasks.

Method: Uses multi-target learning with a frozen Whisper ASR model-based speech prior for acoustic-aware calibration, allowing joint learning of holistic and trait-level objectives without handcrafted features. Processes entire response sessions coherently.

Result: Outperforms previous state-of-the-art cascaded system on Speak & Improve benchmark, excels at predicting holistic oral proficiency, and exhibits robust cross-part generalization.

Conclusion: Produces a compact deployable grader tailored for Computer Assisted Language Learning applications, demonstrating superior performance through session-level processing and multimodal foundation model approach.

Abstract: Spoken Language Assessment (SLA) estimates a learner’s oral proficiency from spontaneous speech. The growing population of L2 English speakers has intensified the demand for reliable SLA, a critical component of Computer Assisted Language Learning (CALL). Existing efforts often rely on cascaded pipelines, which are prone to error propagation, or end-to-end models that often operate on a short audio window, which might miss discourse-level evidence. This paper introduces a novel multimodal foundation model approach that performs session-level evaluation in a single pass. Our approach couples multi-target learning with a frozen, Whisper ASR model-based speech prior for acoustic-aware calibration, allowing for jointly learning holistic and trait-level objectives of SLA without resorting to handcrafted features. By coherently processing the entire response session of an L2 speaker, the model excels at predicting holistic oral proficiency. Experiments conducted on the Speak & Improve benchmark demonstrate that our proposed approach outperforms the previous state-of-the-art cascaded system and exhibits robust cross-part generalization, producing a compact deployable grader that is tailored for CALL applications.

Method: Proposes TVS framework with verbalization step to translate thoughts into speech-ready text, and introduces ReVerT - a latency-efficient verbalizer using incremental and asynchronous summarization.

Result: Experiments show enhanced speech naturalness and conciseness with minimal impact on reasoning performance across multiple benchmarks.

Conclusion: The framework successfully preserves LLM reasoning capacity while improving spoken delivery quality, with code and dataset publicly available.

Abstract: Spoken dialogue systems increasingly employ large language models (LLMs) to leverage their advanced reasoning capabilities. However, direct application of LLMs in spoken communication often yield suboptimal results due to mismatches between optimal textual and verbal delivery. While existing approaches adapt LLMs to produce speech-friendly outputs, their impact on reasoning performance remains underexplored. In this work, we propose Think-Verbalize-Speak, a framework that decouples reasoning from spoken delivery to preserve the full reasoning capacity of LLMs. Central to our method is verbalizing, an intermediate step that translates thoughts into natural, speech-ready text. We also introduce ReVerT, a latency-efficient verbalizer based on incremental and asynchronous summarization. Experiments across multiple benchmarks show that our method enhances speech naturalness and conciseness with minimal impact on reasoning. The project page with the dataset and the source code is available at https://yhytoto12.github.io/TVS-ReVerT

Method: DeCE separates precision (factual accuracy and relevance) and recall (coverage of required concepts), using instance-specific criteria automatically extracted from gold answer requirements. It is model-agnostic and domain-general.

Result: DeCE achieves substantially stronger correlation with expert judgments (r=0.78) compared to traditional metrics (r=0.12), pointwise LLM scoring (r=0.35), and modern multidimensional evaluators (r=0.48). Only 11.95% of LLM-generated criteria required expert revision.

Conclusion: DeCE offers an interpretable and actionable LLM evaluation framework in expert domains, revealing trade-offs between generalist models (favor recall) and specialized models (favor precision).

Abstract: Evaluating long-form answers in high-stakes domains such as law or medicine remains a fundamental challenge. Standard metrics like BLEU and ROUGE fail to capture semantic correctness, and current LLM-based evaluators often reduce nuanced aspects of answer quality into a single undifferentiated score. We introduce DeCE, a decomposed LLM evaluation framework that separates precision (factual accuracy and relevance) and recall (coverage of required concepts), using instance-specific criteria automatically extracted from gold answer requirements. DeCE is model-agnostic and domain-general, requiring no predefined taxonomies or handcrafted rubrics. We instantiate DeCE to evaluate different LLMs on a real-world legal QA task involving multi-jurisdictional reasoning and citation grounding. DeCE achieves substantially stronger correlation with expert judgments ($r=0.78$), compared to traditional metrics ($r=0.12$), pointwise LLM scoring ($r=0.35$), and modern multidimensional evaluators ($r=0.48$). It also reveals interpretable trade-offs: generalist models favor recall, while specialized models favor precision. Importantly, only 11.95% of LLM-generated criteria required expert revision, underscoring DeCE’s scalability. DeCE offers an interpretable and actionable LLM evaluation framework in expert domains.

Method: Proposes Differentiable Expert Pruning (DiEP) that transforms global discrete search space into continuous space, jointly learns inter-layer importance, and enables adaptive gradient-based pruning to handle exponentially growing non-uniform expert combinations.

Result: Extensive experiments on five advanced MoE models show DiEP retains ~92% of original performance on Mixtral 8x7B with only half the experts, outperforming other pruning methods by up to 7.1% on MMLU dataset.

Conclusion: DiEP effectively addresses MoE model scaling challenges by capturing varying redundancy across layers through differentiable non-uniform pruning, demonstrating superior performance across various NLP tasks.

Abstract: Despite the significant breakthrough of Mixture-of-Experts (MoE), the increasing scale of these MoE models presents huge memory and storage challenges. Existing MoE pruning methods, which involve reducing parameter size with a uniform sparsity across all layers, often lead to suboptimal outcomes and performance degradation due to varying expert redundancy in different MoE layers. To address this, we propose a non-uniform pruning strategy, dubbed \textbf{Di}fferentiable \textbf{E}xpert \textbf{P}runing (\textbf{DiEP}), which adaptively adjusts pruning rates at the layer level while jointly learning inter-layer importance, effectively capturing the varying redundancy across different MoE layers. By transforming the global discrete search space into a continuous one, our method handles exponentially growing non-uniform expert combinations, enabling adaptive gradient-based pruning. Extensive experiments on five advanced MoE models demonstrate the efficacy of our method across various NLP tasks. Notably, \textbf{DiEP} retains around 92% of original performance on Mixtral 8$\times$7B with only half the experts, outperforming other pruning methods by up to 7.1% on the challenging MMLU dataset.

Method: Used a novel multilingual evaluation dataset to test DeepSeek v3, GPT-4o, Qwen3-32B, GPT-4o-mini, and Llama-3.1-8B via LLM-as-Judge and human annotations. Fine-tuned Llama-3.1-8B with Direct Preference Optimization (DPO).

Result: Current LLMs struggle to resolve ambiguity effectively, showing limited commonsense reasoning. Simplification prompts drastically reduce commonsense reasoning and diverse response strategies. DPO fine-tuning substantially improves ambiguity resolution across all request types.

Conclusion: Advanced fine-tuning is needed to improve LLMs’ handling of ambiguity and ensure robust performance across diverse communication styles, as current models show significant limitations in referential ambiguity resolution.

Abstract: Ambiguous words or underspecified references require interlocutors to resolve them, often by relying on shared context and commonsense knowledge. Therefore, we systematically investigate whether Large Language Models (LLMs) can leverage commonsense to resolve referential ambiguity in multi-turn conversations and analyze their behavior when ambiguity persists. Further, we study how requests for simplified language affect this capacity. Using a novel multilingual evaluation dataset, we test DeepSeek v3, GPT-4o, Qwen3-32B, GPT-4o-mini, and Llama-3.1-8B via LLM-as-Judge and human annotations. Our findings indicate that current LLMs struggle to resolve ambiguity effectively: they tend to commit to a single interpretation or cover all possible references, rather than hedging or seeking clarification. This limitation becomes more pronounced under simplification prompts, which drastically reduce the use of commonsense reasoning and diverse response strategies. Fine-tuning Llama-3.1-8B with Direct Preference Optimization substantially improves ambiguity resolution across all request types. These results underscore the need for advanced fine-tuning to improve LLMs’ handling of ambiguity and to ensure robust performance across diverse communication styles.

Method: CodeRAG framework with three core components: 1) Log probability guided query construction, 2) Multi-path code retrieval, and 3) Preference-aligned BestFit reranking to identify relevant knowledge for retrieval-augmented code completion.

Result: Extensive experiments on ReccEval and CCEval benchmarks show CodeRAG significantly and consistently outperforms state-of-the-art methods.

Conclusion: CodeRAG effectively addresses key limitations in repository-level code completion and demonstrates superior performance through its innovative query construction, retrieval, and reranking approach.

Abstract: Repository-level code completion automatically predicts the unfinished code based on the broader information from the repository. Recent strides in Code Large Language Models (code LLMs) have spurred the development of repository-level code completion methods, yielding promising results. Nevertheless, they suffer from issues such as inappropriate query construction, single-path code retrieval, and misalignment between code retriever and code LLM. To address these problems, we introduce CodeRAG, a framework tailored to identify relevant and necessary knowledge for retrieval-augmented repository-level code completion. Its core components include log probability guided query construction, multi-path code retrieval, and preference-aligned BestFit reranking. Extensive experiments on benchmarks ReccEval and CCEval demonstrate that CodeRAG significantly and consistently outperforms state-of-the-art methods. The implementation of CodeRAG is available at https://github.com/KDEGroup/CodeRAG.

Method: Proposed CultureScope framework based on cultural iceberg theory, with a dimensional schema (3 layers, 140 dimensions) that enables automated construction of culture-specific knowledge bases and evaluation datasets for any language/culture.

Result: Experimental results show the method effectively evaluates cultural understanding, revealing that existing LLMs lack comprehensive cultural competence and that multilingual data alone doesn’t necessarily improve cultural understanding.

Conclusion: CultureScope provides a scalable, theory-grounded approach to cultural evaluation in LLMs, demonstrating current models’ cultural limitations and the need for more sophisticated cultural understanding beyond just multilingual training.

Abstract: As large language models (LLMs) are increasingly deployed in diverse cultural environments, evaluating their cultural understanding capability has become essential for ensuring trustworthy and culturally aligned applications. However, most existing benchmarks lack comprehensiveness and are challenging to scale and adapt across different cultural contexts, because their frameworks often lack guidance from well-established cultural theories and tend to rely on expert-driven manual annotations. To address these issues, we propose CultureScope, the most comprehensive evaluation framework to date for assessing cultural understanding in LLMs. Inspired by the cultural iceberg theory, we design a novel dimensional schema for cultural knowledge classification, comprising 3 layers and 140 dimensions, which guides the automated construction of culture-specific knowledge bases and corresponding evaluation datasets for any given languages and cultures. Experimental results demonstrate that our method can effectively evaluate cultural understanding. They also reveal that existing large language models lack comprehensive cultural competence, and merely incorporating multilingual data does not necessarily enhance cultural understanding. All code and data files are available at https://github.com/HoganZinger/Culture

Method: ZeroRepo uses a three-stage graph-driven framework: proposal-level planning, implementation-level refinement to construct the Repository Planning Graph (RPG), followed by graph-guided code generation with test validation.

Result: ZeroRepo generates repositories averaging nearly 36K LOC (3.9x stronger baseline), achieves 81.5% functional coverage and 69.7% pass rate, exceeding Claude Code by 27.3 and 35.8 percentage points respectively.

Conclusion: RPG effectively models complex dependencies, enables sophisticated planning through near-linear scaling, and enhances LLM understanding of repositories, making it a promising approach for repository generation from scratch.

Abstract: Large language models excel at function- and file-level code generation, yet generating complete repositories from scratch remains a fundamental challenge. This process demands coherent and reliable planning across proposal- and implementation-level stages, while natural language, due to its ambiguity and verbosity, is ill-suited for faithfully representing complex software structures. To address this, we introduce the Repository Planning Graph (RPG), a persistent representation that unifies proposal- and implementation-level planning by encoding capabilities, file structures, data flows, and functions in one graph. RPG replaces ambiguous natural language with an explicit blueprint, enabling long-horizon planning and scalable repository generation. Building on RPG, we develop ZeroRepo, a graph-driven framework for repository generation from scratch. It operates in three stages: proposal-level planning and implementation-level refinement to construct the graph, followed by graph-guided code generation with test validation. To evaluate this setting, we construct RepoCraft, a benchmark of six real-world projects with 1,052 tasks. On RepoCraft, ZeroRepo produces repositories averaging nearly 36K LOC, roughly 3.9$\times$ the strongest baseline (Claude Code) and about 64$\times$ other baselines. It attains 81.5% functional coverage and a 69.7% pass rate, exceeding Claude Code by 27.3 and 35.8 percentage points, respectively. Further analysis shows that RPG models complex dependencies, enables progressively more sophisticated planning through near-linear scaling, and enhances LLM understanding of repositories, thereby accelerating agent localization.

Method: Developed a text simplification pipeline combining pretrained BERT representation model with morphological features to generate grammatically correct and semantically appropriate word-level simplifications.

Result: Presented the first automatic lexical simplification system specifically designed for Turkish language.

Conclusion: The proposed approach addresses Turkish’s unique linguistic challenges and resource limitations through a BERT-based pipeline with morphological integration.

Abstract: In this paper, we present the first automatic lexical simplification system for the Turkish language. Recent text simplification efforts rely on manually crafted simplified corpora and comprehensive NLP tools that can analyse the target text both in word and sentence levels. Turkish is a morphologically rich agglutinative language that requires unique considerations such as the proper handling of inflectional cases. Being a low-resource language in terms of available resources and industrial-strength tools, it makes the text simplification task harder to approach. We present a new text simplification pipeline based on pretrained representation model BERT together with morphological features to generate grammatically correct and semantically appropriate word-level simplifications.

Method: The approach fits transformer-based model embeddings into a stochastic process to create ordered latent representations from unordered outputs, establishing a “clustered-to-temporal ordered” mapping in high-dimensional space.

Result: The method demonstrates improved performance on temporal consistency evaluation (Shuffle tasks) and AI-generated content detection, with the latent structure aligning well with natural language properties.

Conclusion: Stochastic process modeling of transformer embeddings provides an effective framework for evaluating language model outputs, offering both intuitive and quantitative benefits for sequence evaluation tasks.

Abstract: Autoregressive generative models play a key role in various language tasks, especially for modeling and evaluating long text sequences. While recent methods leverage stochastic representations to better capture sequence dynamics, encoding both temporal and structural dependencies and utilizing such information for evaluation remains challenging. In this work, we observe that fitting transformer-based model embeddings into a stochastic process yields ordered latent representations from originally unordered model outputs. Building on this insight and prior work, we theoretically introduce a novel likelihood-based evaluation metric BBScoreV2. Empirically, we demonstrate that the stochastic latent space induces a “clustered-to-temporal ordered” mapping of language model representations in high-dimensional space, offering both intuitive and quantitative support for the effectiveness of BBScoreV2. Furthermore, this structure aligns with intrinsic properties of natural language and enhances performance on tasks such as temporal consistency evaluation (e.g., Shuffle tasks) and AI-generated content detection.

Method: Database-Augmented Query representation (DAQu) framework expands original queries with various query-related metadata across multiple database tables, using graph-based set encoding to handle large feature sets without order constraints while considering feature hierarchies.

Result: DAQu significantly enhances overall retrieval performance across diverse retrieval scenarios compared to relevant baselines.

Conclusion: The proposed DAQu framework effectively improves document retrieval by leveraging database metadata through graph-based encoding, demonstrating superior performance over existing query expansion methods.

Abstract: Information retrieval models that aim to search for documents relevant to a query have shown multiple successes, which have been applied to diverse tasks. Yet, the query from the user is oftentimes short, which challenges the retrievers to correctly fetch relevant documents. To tackle this, previous studies have proposed expanding the query with a couple of additional (user-related) features related to it. However, they may be suboptimal to effectively augment the query, and there is plenty of other information available to augment it in a relational database. Motivated by this fact, we present a novel retrieval framework called Database-Augmented Query representation (DAQu), which augments the original query with various (query-related) metadata across multiple tables. In addition, as the number of features in the metadata can be very large and there is no order among them, we encode them with the graph-based set-encoding strategy, which considers hierarchies of features in the database without order. We validate our DAQu in diverse retrieval scenarios, demonstrating that it significantly enhances overall retrieval performance over relevant baselines.

Method: Conducted a snippet-based online survey with 17 peer reviewers from top-tier HCI conferences to compare human and AI-augmented writing samples.

Result: AI-augmented writing improves readability, language diversity, and informativeness but often lacks research details and reflective insights. Reviewers couldn’t reliably distinguish between human and AI writing but maintained consistent evaluation standards.

Conclusion: Reviewer guidelines should promote impartial evaluations focusing on research quality rather than AI usage, and researchers must maintain authorship control when using GenAI assistance.

Abstract: Generative AI (GenAI) use in research writing is growing fast. However, it is unclear how peer reviewers recognize or misjudge AI-augmented manuscripts. To investigate the impact of AI-augmented writing on peer reviews, we conducted a snippet-based online survey with 17 peer reviewers from top-tier HCI conferences. Our findings indicate that while AI-augmented writing improves readability, language diversity, and informativeness, it often lacks research details and reflective insights from authors. Reviewers consistently struggled to distinguish between human and AI-augmented writing but their judgements remained consistent. They noted the loss of a “human touch” and subjective expressions in AI-augmented writing. Based on our findings, we advocate for reviewer guidelines that promote impartial evaluations of submissions, regardless of any personal biases towards GenAI. The quality of the research itself should remain a priority in reviews, regardless of any preconceived notions about the tools used to create it. We emphasize that researchers must maintain their authorship and control over the writing process, even when using GenAI’s assistance.

Method: Developed ConfReady using retrieval-augmented generation (RAG), curated a dataset of 1,975 ACL checklist responses, analyzed human answer problems, and benchmarked RAG and LLM-based systems.

Result: Created an open-source tool (AGPL-3.0 license) with user interface and PyPI package that helps authors complete conference checklists more accurately.

Conclusion: ConfReady provides an effective solution to improve checklist completion accuracy and helps authors better reflect on their research practices before paper submission.

Abstract: The ARR Responsible NLP Research checklist website states that the “checklist is designed to encourage best practices for responsible research, addressing issues of research ethics, societal impact and reproducibility.” Answering the questions is an opportunity for authors to reflect on their work and make sure any shared scientific assets follow best practices. Ideally, considering a checklist before submission can favorably impact the writing of a research paper. However, previous research has shown that self-reported checklist responses don’t always accurately represent papers. In this work, we introduce ConfReady, a retrieval-augmented generation (RAG) application that can be used to empower authors to reflect on their work and assist authors with conference checklists. To evaluate checklist assistants, we curate a dataset of 1,975 ACL checklist responses, analyze problems in human answers, and benchmark RAG and Large Language Model (LM) based systems on an evaluation subset. Our code is released under the AGPL-3.0 license on GitHub, with documentation covering the user interface and PyPI package.

Method: Proposes DynamicNER dataset with dynamic categorization where entities can have different types in different contexts, and introduces CascadeNER - a two-stage NER method using lightweight LLMs.

Result: Experiments show DynamicNER serves as a robust benchmark for LLM-based NER methods, and CascadeNER achieves higher accuracy on fine-grained tasks while requiring fewer computational resources.

Conclusion: DynamicNER addresses limitations of traditional NER datasets and enables better evaluation of LLM-based methods’ capabilities, while CascadeNER demonstrates efficient fine-grained NER performance.

Abstract: The advancements of Large Language Models (LLMs) have spurred a growing interest in their application to Named Entity Recognition (NER) methods. However, existing datasets are primarily designed for traditional machine learning methods and are inadequate for LLM-based methods, in terms of corpus selection and overall dataset design logic. Moreover, the prevalent fixed and relatively coarse-grained entity categorization in existing datasets fails to adequately assess the superior generalization and contextual understanding capabilities of LLM-based methods, thereby hindering a comprehensive demonstration of their broad application prospects. To address these limitations, we propose DynamicNER, the first NER dataset designed for LLM-based methods with dynamic categorization, introducing various entity types and entity type lists for the same entity in different context, leveraging the generalization of LLM-based NER better. The dataset is also multilingual and multi-granular, covering 8 languages and 155 entity types, with corpora spanning a diverse range of domains. Furthermore, we introduce CascadeNER, a novel NER method based on a two-stage strategy and lightweight LLMs, achieving higher accuracy on fine-grained tasks while requiring fewer computational resources. Experiments show that DynamicNER serves as a robust and effective benchmark for LLM-based NER methods. Furthermore, we also conduct analysis for traditional methods and LLM-based methods on our dataset. Our code and dataset are openly available at https://github.com/Astarojth/DynamicNER.

Method: Use ICL-based teacher to generate pseudo-labels on unlabeled queries, while student predicts using only query input and updates lightweight adapter. This captures demonstration effects in reusable form.

Result: Method improves generalization, stability, and efficiency across in-domain and out-of-domain tasks, surpassing standard ICL and prior disentanglement methods. Student often outperforms teacher through pseudo-label correction.

Conclusion: Proposed approach enables efficient inference while remaining composable with new demonstrations, demonstrating weak-to-strong generalization effect.

Abstract: In-context learning (ICL) enables large language models to perform few-shot learning by conditioning on labeled examples in the prompt. Despite its flexibility, ICL suffers from instability – especially as prompt length increases with more demonstrations. To address this, we treat ICL as a source of weak supervision and propose a parameter-efficient method that disentangles demonstration-induced latent shifts from those of the query. An ICL-based teacher generates pseudo-labels on unlabeled queries, while a student predicts them using only the query input, updating a lightweight adapter. This captures demonstration effects in a compact, reusable form, enabling efficient inference while remaining composable with new demonstrations. Although trained on noisy teacher outputs, the student often outperforms its teacher through pseudo-label correction and coverage expansion, consistent with the weak-to-strong generalization effect. Empirically, our method improves generalization, stability, and efficiency across both in-domain and out-of-domain tasks, surpassing standard ICL and prior disentanglement methods.

Method: Extract-Refine-Retrieve-Read framework: extracts parametric knowledge from LLMs, uses specialized query optimizer to refine queries, retrieves only pertinent information. Includes trainable scheme with smaller tunable model as query optimizer refined through knowledge distillation.

Result: ERRR consistently outperforms existing baselines on various QA datasets with different retrieval systems, proving versatile and cost-effective.

Conclusion: ERRR is an effective framework for improving RAG system utility and accuracy through optimized query processing and knowledge retrieval.

Abstract: We introduce the \textit{Extract-Refine-Retrieve-Read} (ERRR) framework, a novel approach designed to bridge the pre-retrieval information gap in Retrieval-Augmented Generation (RAG) systems through query optimization tailored to meet the specific knowledge requirements of Large Language Models (LLMs). Unlike conventional query optimization techniques used in RAG, the ERRR framework begins by extracting parametric knowledge from LLMs, followed by using a specialized query optimizer for refining these queries. This process ensures the retrieval of only the most pertinent information essential for generating accurate responses. Moreover, to enhance flexibility and reduce computational costs, we propose a trainable scheme for our pipeline that utilizes a smaller, tunable model as the query optimizer, which is refined through knowledge distillation from a larger teacher model. Our evaluations on various question-answering (QA) datasets and with different retrieval systems show that ERRR consistently outperforms existing baselines, proving to be a versatile and cost-effective module for improving the utility and accuracy of RAG systems.

Method: Streaming-VR enables on-the-fly verification and correction of tokens as they are generated, using another LLM to check and refine each token subset in real-time during response construction.

Result: Comprehensive evaluations on multiple datasets show that Streaming-VR enhances factual accuracy of LLMs while providing more efficient verification compared to prior refinement methods.

Conclusion: The proposed streaming verification approach effectively addresses the limitations of post-generation verification by enabling real-time error detection and correction during LLM response generation.

Abstract: Large language models (LLMs) have shown remarkable performance across a wide range of natural language tasks. However, a critical challenge remains in that they sometimes generate factually incorrect answers. To address this, while many previous work has focused on identifying errors in their generation and further refining them, they are slow in deployment since they are designed to verify the response from LLMs only after their entire generation (from the first to last tokens) is done. Further, we observe that once LLMs generate incorrect tokens early on, there is a higher likelihood that subsequent tokens will also be factually incorrect. To this end, in this work, we propose Streaming-VR (Streaming Verification and Refinement), a novel approach designed to enhance the efficiency of verification and refinement of LLM outputs. Specifically, the proposed Streaming-VR enables on-the-fly verification and correction of tokens as they are being generated, similar to a streaming process, ensuring that each subset of tokens is checked and refined in real-time by another LLM as the LLM constructs its response. Through comprehensive evaluations on multiple datasets, we demonstrate that our approach not only enhances the factual accuracy of LLMs, but also offers a more efficient solution compared to prior refinement methods.

Method: Stacked Multi-Model Reasoning (SMMR) uses multiple LLMs and specialized smaller models as coequal experts, with early layers handling discrete subtasks and later layers integrating outputs through advanced long-context models.

Result: Evaluation on DAIC-WOZ depression-screening dataset and 48 case studies showed consistent improvements over single-model baselines in accuracy, F1-score, and PHQ-8 error reduction.

Conclusion: Multi-expert frameworks like SMMR enhance reliability in high-stakes mental health assessments by mitigating hallucinations and capturing clinical nuances.

Abstract: Long-form mental health assessments pose unique challenges for large language models (LLMs), which often exhibit hallucinations or inconsistent reasoning when handling extended, domain-specific contexts. We introduce Stacked Multi-Model Reasoning (SMMR), a layered framework that leverages multiple LLMs and specialized smaller models as coequal ’experts’. Early layers isolate short, discrete subtasks, while later layers integrate and refine these partial outputs through more advanced long-context models. We evaluate SMMR on the DAIC-WOZ depression-screening dataset and 48 curated case studies with psychiatric diagnoses, demonstrating consistent improvements over single-model baselines in terms of accuracy, F1-score, and PHQ-8 error reduction. By harnessing diverse ‘second opinions’, SMMR mitigates hallucinations, captures subtle clinical nuances, and enhances reliability in high-stakes mental health assessments. Our findings underscore the value of multi-expert frameworks for more trustworthy AI-driven screening.

Method: The approach taps into human psychological principles to seamlessly modify product descriptions, making manipulations hard to detect. It investigates cognitive biases as black-box adversarial strategies and evaluates their effects across LLMs of varying scales.

Result: Certain biases like social proof consistently boost product recommendation rate and ranking, while others like scarcity and exclusivity surprisingly reduce product visibility. Cognitive biases are deeply embedded in state-of-the-art LLMs.

Conclusion: Cognitive biases lead to highly unpredictable behavior in LLM-based product recommendations and pose significant challenges for effective mitigation strategies in commercial applications.

Abstract: The advent of Large Language Models (LLMs) has revolutionized product recommenders, yet their susceptibility to adversarial manipulation poses critical challenges, particularly in real-world commercial applications. Our approach is the first one to tap into human psychological principles, seamlessly modifying product descriptions, making such manipulations hard to detect. In this work, we investigate cognitive biases as black-box adversarial strategies, drawing parallels between their effects on LLMs and human purchasing behavior. Through extensive evaluation across models of varying scale, we find that certain biases, such as social proof, consistently boost product recommendation rate and ranking, while others, like scarcity and exclusivity, surprisingly reduce visibility. Our results demonstrate that cognitive biases are deeply embedded in state-of-the-art LLMs, leading to highly unpredictable behavior in product recommendations and posing significant challenges for effective mitigation.

Method: Introduces an adaptive self-improvement agentic system and constructs a benchmark of typical ML libraries to evaluate ASPL code generation using both open and closed-source LLMs.

Result: The system shows improvements of up to 3.9x over baseline single LLM performance in generating ML libraries using ASPLs.

Conclusion: The proposed agentic system effectively addresses the challenges of using LLMs for ASPL-based ML library generation, demonstrating significant performance improvements.

Abstract: ML libraries, often written in architecture-specific programming languages (ASPLs) that target domain-specific architectures, are key to efficient ML systems. However, writing these high-performance ML libraries is challenging because it requires expert knowledge of ML algorithms and the ASPL. Large language models (LLMs), on the other hand, have shown general coding capabilities. However, challenges remain when using LLMs for generating ML libraries using ASPLs because 1) this task is complicated even for experienced human programmers and 2) there are limited code examples because of the esoteric and evolving nature of ASPLs. Therefore, LLMs need complex reasoning with limited data in order to complete this task. To address these challenges, we introduce an adaptive self-improvement agentic system. In order to evaluate the effectiveness of our system, we construct a benchmark of a typical ML library and generate ASPL code with both open and closed-source LLMs on this benchmark. Our results show improvements of up to $3.9\times$ over a baseline single LLM.

Method: Developed Fact-or-Fair benchmark with objective (fact-based) and subjective (fairness-based) queries constructed from 19 statistics, grounded in cognitive psychology biases like representativeness bias, attribution bias, and ingroup-outgroup bias.

Result: Experiments across ten frontier models revealed different levels of fact-fair trade-offs, showing how models often misalign factual accuracy with normative fairness.

Conclusion: The paper provides both a theoretical framework and practical benchmark to advance responsible AI model assessments by clearly distinguishing between factual correctness and social fairness dimensions.

Abstract: Recent failures such as Google Gemini generating people of color in Nazi-era uniforms illustrate how AI outputs can be factually plausible yet socially harmful. AI models are increasingly evaluated for “fairness,” yet existing benchmarks often conflate two fundamentally different dimensions: factual correctness and normative fairness. A model may generate responses that are factually accurate but socially unfair, or conversely, appear fair while distorting factual reality. We argue that identifying the boundary between fact and fair is essential for meaningful fairness evaluation. We introduce Fact-or-Fair, a benchmark with (i) objective queries aligned with descriptive, fact-based judgments, and (ii) subjective queries aligned with normative, fairness-based judgments. Our queries are constructed from 19 statistics and are grounded in cognitive psychology, drawing on representativeness bias, attribution bias, and ingroup-outgroup bias to explain why models often misalign fact and fairness. Experiments across ten frontier models reveal different levels of fact-fair trade-offs. By reframing fairness evaluation, we provide both a new theoretical lens and a practical benchmark to advance the responsible model assessments. Our test suite is publicly available at https://github.com/uclanlp/Fact-or-Fair.

Method: Uses Heim and Kratzer’s syntax-based compositional semantic rules with lambda calculus, featuring abstract types, templated rules, and dynamic context interpretation. Logical forms are executed via constraint logic programming.

Result: Achieves 100% accuracy on BIG-bench’s logical deduction task and 88% on a syntactically simplified subset of AR-LSAT, outperforming the o1-preview LLM baseline.

Conclusion: FSLI demonstrates the potential of principled, interpretable symbolic systems for logical deduction in NLP, offering an alternative to neural approaches.

Abstract: We develop a system for solving logical deduction one-dimensional ordering problems by transforming natural language premises and candidate statements into first-order logic. Building on Heim and Kratzer’s syntax-based compositional semantic rules which utilizes lambda calculus, we develop a semantic parsing algorithm with abstract types, templated rules, and a dynamic component for interpreting entities within a context constructed from the input. The resulting logical forms are executed via constraint logic programming to determine which candidate statements can be logically deduced from the premises. The symbolic system, the Formal Semantic Logic Inferer (FSLI), provides a formally grounded, linguistically driven system for natural language logical deduction. We evaluate it on both synthetic and derived logical deduction problems. FSLI achieves 100% accuracy on BIG-bench’s logical deduction task and 88% on a syntactically simplified subset of AR-LSAT outperforming an LLM baseline, o1-preview. While current research in natural language reasoning emphasizes neural language models, FSLI highlights the potential of principled, interpretable systems for symbolic logical deduction in NLP.

Method: Randomly select a small proportion of model parameters to train while freezing all others, without any additional structural assumptions like low-rank matrices.

Result: The proposed method performs competitively with LoRA when using a similar number of trainable parameters.

Conclusion: What matters most for PEFT performance is the number of trainable parameters, not the specific adapter structure or prior assumptions.

Abstract: Full fine-tuning of large language models for alignment and task adaptation has become prohibitively expensive as models have grown in size. Parameter-Efficient Fine-Tuning (PEFT) methods aim at significantly reducing the computational and memory resources needed for fine-tuning these models by only training on a small number of parameters instead of all model parameters. Currently, the most popular PEFT method is the Low-Rank Adaptation (LoRA), which freezes the parameters of the model and introduces a small set of trainable parameters in the form of low-rank matrices. We propose simply reducing the number of trainable parameters by randomly selecting a small proportion of the model parameters to train on, while fixing all other parameters, without any additional prior assumptions such as low-rank structures. In this paper, we compare the efficiency and performance of our proposed approach to other PEFT methods as well as full parameter fine-tuning. We find our method to be competitive with LoRA when using a similar number of trainable parameters. Our findings suggest that what truly matters for a PEFT technique to perform well is not necessarily the specific adapter structure, but rather the number of trainable parameters being used.

Method: Created MultiOCR-QA dataset with 50K QA pairs across three languages from OCR-ed historical documents containing various OCR noise levels and types. Evaluated state-of-the-art LLMs under different OCR error conditions focusing on three major error types.

Result: QA systems are highly vulnerable to OCR-induced errors and perform poorly on noisy OCR text compared to clean text.

Conclusion: Current QA approaches have significant limitations in handling OCR noise, highlighting the need for more noise-resilient systems in historical digitization contexts.

Abstract: Optical Character Recognition (OCR) plays a crucial role in digitizing historical and multilingual documents, yet OCR errors - imperfect extraction of text, including character insertion, deletion, and substitution can significantly impact downstream tasks like question-answering (QA). In this work, we conduct a comprehensive analysis of how OCR-induced noise affects the performance of Multilingual QA Systems. To support this analysis, we introduce a multilingual QA dataset MultiOCR-QA, comprising 50K question-answer pairs across three languages, English, French, and German. The dataset is curated from OCR-ed historical documents, which include different levels and types of OCR noise. We then evaluate how different state-of-the-art Large Language Models (LLMs) perform under different error conditions, focusing on three major OCR error types. Our findings show that QA systems are highly prone to OCR-induced errors and perform poorly on noisy OCR text. By comparing model performance on clean versus noisy texts, we provide insights into the limitations of current approaches and emphasize the need for more noise-resilient QA systems in historical digitization contexts.

Method: The paper introduces a taxonomy of LLM Ensemble, provides in-depth classification of methods into three categories, reviews relevant methods, and discusses benchmarks and applications.

Result: A comprehensive systematic review of LLM Ensemble developments with curated resources and future research directions.

Conclusion: LLM Ensemble represents an important advancement in leveraging multiple language models, and the paper provides foundational taxonomy and directions for future research in this emerging field.

Abstract: LLM Ensemble – which involves the comprehensive use of multiple large language models (LLMs), each aimed at handling user queries during downstream inference, to benefit from their individual strengths – has gained substantial attention recently. The widespread availability of LLMs, coupled with their varying strengths and out-of-the-box usability, has profoundly advanced the field of LLM Ensemble. This paper presents the first systematic review of recent developments in LLM Ensemble. First, we introduce our taxonomy of LLM Ensemble and discuss several related research problems. Then, we provide a more in-depth classification of the methods under the broad categories of “ensemble-before-inference, ensemble-during-inference, ensemble-after-inference’’, and review all relevant methods. Finally, we introduce related benchmarks and applications, summarize existing studies, and suggest several future research directions. A curated list of papers on LLM Ensemble is available at https://github.com/junchenzhi/Awesome-LLM-Ensemble.

Method: The authors created KatFish dataset with human-written and LLM-generated Korean text across three genres. They analyzed linguistic differences through spacing patterns, part-of-speech diversity, and comma usage, then developed KatFishNet specifically designed for Korean language characteristics.

Result: KatFishNet achieved an average of 19.78% higher AUROC compared to the best-performing existing detection method, demonstrating significant improvement in detecting LLM-generated Korean text.

Conclusion: The study highlights the importance of language-specific approaches for LLM-generated text detection and provides the first comprehensive benchmark and effective detection method for Korean, with potential applications for other languages with unique linguistic characteristics.

Abstract: The rapid advancement of large language models (LLMs) increases the difficulty of distinguishing between human-written and LLM-generated text. Detecting LLM-generated text is crucial for upholding academic integrity, preventing plagiarism, protecting copyrights, and ensuring ethical research practices. Most prior studies on detecting LLM-generated text focus primarily on English text. However, languages with distinct morphological and syntactic characteristics require specialized detection approaches. Their unique structures and usage patterns can hinder the direct application of methods primarily designed for English. Among such languages, we focus on Korean, which has relatively flexible spacing rules, a rich morphological system, and less frequent comma usage compared to English. We introduce KatFish, the first benchmark dataset for detecting LLM-generated Korean text. The dataset consists of text written by humans and generated by four LLMs across three genres. By examining spacing patterns, part-of-speech diversity, and comma usage, we illuminate the linguistic differences between human-written and LLM-generated Korean text. Building on these observations, we propose KatFishNet, a detection method specifically designed for the Korean language. KatFishNet achieves an average of 19.78% higher AUROC compared to the best-performing existing detection method. Our code and data are available at https://github.com/Shinwoo-Park/detecting_llm_generated_korean_text_through_linguistic_analysis.

Method: A three-stage framework combining local differential privacy with composition theorem via group text rewriting, integrating document-level and word-level information while exploiting in-context learning to bridge local and global DP mechanisms at individual data point level.

Result: Experiments on CommonSense QA and DocVQA show DP-GTR outperforms existing approaches with superior privacy-utility trade-off, and it’s compatible with existing rewriting techniques as a plug-in enhancement.

Conclusion: DP-GTR effectively addresses the limitations of current prompt privacy methods by providing a comprehensive framework that improves both privacy protection and utility, making it practical for real-world LLM applications.

Abstract: Prompt privacy is crucial, especially when using online large language models (LLMs), due to the sensitive information often contained within prompts. While LLMs can enhance prompt privacy through text rewriting, existing methods primarily focus on document-level rewriting, neglecting the rich, multi-granular representations of text. This limitation restricts LLM utilization to specific tasks, overlooking their generalization and in-context learning capabilities, thus hindering practical application. To address this gap, we introduce DP-GTR, a novel three-stage framework that leverages local differential privacy (DP) and the composition theorem via group text rewriting. DP-GTR is the first framework to integrate both document-level and word-level information while exploiting in-context learning to simultaneously improve privacy and utility, effectively bridging local and global DP mechanisms at the individual data point level. Experiments on CommonSense QA and DocVQA demonstrate that DP-GTR outperforms existing approaches, achieving a superior privacy-utility trade-off. Furthermore, our framework is compatible with existing rewriting techniques, serving as a plug-in to enhance privacy protection. Our code is publicly available at github.com/ResponsibleAILab/DP-GTR.

Method: Propose CSO approach that optimizes strategy selection preferences at each dialogue turn. Use Monte Carlo Tree Search to construct ESC-Pro dataset with turn-level strategy-response pairs, then train LLMs on this dataset.

Result: CSO outperforms standard SFT on LLaMA-3.1-8B, Gemma-2-9B, and Qwen2.5-7B models, improving both strategy accuracy and bias mitigation for more empathetic and contextually appropriate responses.

Conclusion: Fine-grained, turn-level preference modeling through CSO is effective for enhancing emotional support conversations in LLMs.

Abstract: The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to emotional needs of users. Existing supervised fine-tuning (SFT) struggles to address these issues, as it rigidly trains models on single gold-standard responses without modeling nuanced strategy trade-offs. To overcome these limitations, we propose Chain-of-Strategy Optimization (CSO), a novel approach that optimizes strategy selection preferences at each dialogue turn. We first leverage Monte Carlo Tree Search to construct ESC-Pro, a high-quality preference dataset with turn-level strategy-response pairs. Training on ESC-Pro with CSO improves both strategy accuracy and bias mitigation, enabling LLMs to generate more empathetic and contextually appropriate responses. Experiments on LLaMA-3.1-8B, Gemma-2-9B, and Qwen2.5-7B demonstrate that CSO outperforms standard SFT, highlighting the efficacy of fine-grained, turn-level preference modeling in ESC.

Method: Built reWordBench to systematically transform reward model inputs in meaning- or ranking-preserving ways. Proposed training reward models to assign similar scores to paraphrases to improve robustness.

Result: State-of-the-art reward models suffer substantial performance degradation even with minor input transformations, sometimes dropping below random accuracy. The proposed robust training method reduces degradation by roughly half for Chat Hard subset and improves alignment utility.

Conclusion: Reward models are brittle to input transformations, but explicit paraphrasing training significantly improves robustness and leads to higher-quality outputs in alignment applications.

Abstract: Reward models have become a staple in modern NLP, serving as not only a scalable text evaluator, but also an indispensable component in many alignment recipes and inference-time algorithms. However, while recent reward models increase performance on standard benchmarks, this may partly be due to overfitting effects, which would confound an understanding of their true capability. In this work, we scrutinize the robustness of reward models and the extent of such overfitting. We build reWordBench, which systematically transforms reward model inputs in meaning- or ranking-preserving ways. We show that state-of-the-art reward models suffer from substantial performance degradation even with minor input transformations, sometimes dropping to significantly below-random accuracy, suggesting brittleness. To improve reward model robustness, we propose to explicitly train them to assign similar scores to paraphrases, and find that this approach also improves robustness to other distinct kinds of transformations. For example, our robust reward model reduces such degradation by roughly half for the Chat Hard subset in RewardBench. Furthermore, when used in alignment, our robust reward models demonstrate better utility and lead to higher-quality outputs, winning in up to 59% of instances against a standardly trained RM.

Method: Proposed MT-RewardTree framework with automatic token-level preference pair generation using approximate Monte Carlo Tree Search, and systematic comparison of reward modeling architectures.

Result: MT-PRM-Qwen-2.5-3B achieves state-of-the-art performance in both token-level and sequence-level evaluation, and enables test-time alignment without additional training while improving hypothesis ensembling.

Conclusion: The work provides valuable insights into reward models in MT research, demonstrating that token-level supervision effectively captures fine-grained preferences and PRMs have practical applications for alignment and ensembling.

Abstract: Process reward models (PRMs) have shown success in complex reasoning tasks for large language models (LLMs). However, their application to machine translation (MT) remains underexplored due to the lack of systematic methodologies and evaluation benchmarks. To address this gap, we introduce \textbf{MT-RewardTree}, a comprehensive framework for constructing, evaluating, and deploying process reward models in MT. Unlike traditional vanilla preference pair construction, we propose a novel method for automatically generating token-level preference pairs using approximate Monte Carlo Tree Search (MCTS), which mitigates the prohibitive cost of human annotation for fine-grained steps. Then, we establish the first MT-specific reward model benchmark and provide a systematic comparison of different reward modeling architectures, revealing that token-level supervision effectively captures fine-grained preferences. Experimental results demonstrate that our MT-PRM-Qwen-2.5-3B achieves state-of-the-art performance in both token-level and sequence-level evaluation given the same input prefix. Furthermore, we showcase practical applications where PRMs enable test-time alignment for LLMs without additional alignment training and significantly improve performance in hypothesis ensembling. Our work provides valuable insights into the role of reward models in MT research. Our code and data are released in \href{https://sabijun.github.io/MT_RewardTreePage/}{https://sabijun.github.io/MT_RewardTreePage}.

Method: Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME) - gradient-free optimization that integrates privacy preservation into the objective function.

Result: Outperforms baselines with up to 45% improvement in task performance on LaMP benchmark, superior privacy-utility trade-off, enhanced robustness to membership inference attacks.

Conclusion: PriME effectively captures user preferences while minimizing privacy risks, demonstrating superior performance and privacy protection compared to state-of-the-art methods.

Abstract: Personalization in language models aims to tailor model behavior to individual users or user groups. Prompt-based methods incorporate user preferences into queries, while training-based methods encode them into model parameters. Model merging has also been explored for personalization under limited data. However, existing methods often fail to directly optimize task-specific utility and lack explicit mechanisms for privacy preservation. To address the limitations, we propose Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME), a novel personalization approach that employs gradient-free methods to directly optimize utility while reducing privacy risks. By integrating privacy preservation into the optimization objective, PriME creates personalized modules that effectively capture target user preferences while minimizing privacy risks for data-sharing users. Experiments on the LaMP benchmark show that PriME consistently outperforms a range of baselines, achieving up to a 45% improvement in task performance. Further analysis demonstrates that PriME achieves a superior privacy-utility trade-off compared to a prior state-of-the-art, with enhanced robustness to membership inference attacks and greater utility in capturing user preferences.

Method: Used OpenAI GPT-4-turbo, o1, and GPT-OSS models to conduct end-to-end relation extraction experiments on seven datasets, employing JSON generation capabilities with both explicit schema definition and structure inference from prompt language.

Result: Zero-shot performances were proximal to fine-tuned methods, but models performed poorly on instances with many relations and made errors on textual mention boundaries. This is the first study comparing GPT-4, o1, and GPT-OSS for end-to-end biomedical relation extraction.

Conclusion: LLMs show promising zero-shot capabilities for biomedical relation extraction, offering competitive performance with reduced curation costs, though with increased compute costs. Addressing identified limitations could improve reliability.

Abstract: Objective: Zero-shot methodology promises to cut down on costs of dataset annotation and domain expertise needed to make use of NLP. Generative large language models trained to align with human goals have achieved high zero-shot performance across a wide variety of tasks. As of yet, it is unclear how well these models perform on biomedical relation extraction (RE). To address this knowledge gap, we explore patterns in the performance of OpenAI LLMs across a diverse sampling of RE tasks. Methods: We use OpenAI GPT-4-turbo and OpenAI’s reasoning models o1 and GPT-OSS to conduct end-to-end RE experiments on seven datasets. We use the JSON generation capabilities of GPT models to generate structured output in two ways: (1) by defining an explicit schema describing the structure of relations, and (2) using a setting that infers the structure from the prompt language. Results: Our work is the first to study and compare the performance of the GPT-4, o1 and GPT-OSS for the end-to-end zero-shot biomedical RE task across a broad array of datasets. We found the zero-shot performances to be proximal to that of fine-tuned methods. The limitations of this approach are that it performs poorly on instances containing many relations and errs on the boundaries of textual mentions. Conclusion: LLMs exhibit promising zero-shot capabilities in complex biomedical RE tasks, offering competitive performance with reduced dataset curation costs and NLP modeling needs but with increased perpetual compute costs. Addressing the limitations we identify could further boost reliability. The code, data, and prompts for all our experiments are publicly available for additional benchmarking by the community: https://github.com/bionlproc/ZeroShotRE

Method: The system features a Persona Generator module, LLM Agent module, and Universal Browser Connector module to automatically generate thousands of simulated users who interactively test websites. It provides a Result Viewer Interface for analyzing qualitative and quantitative data.

Result: Heuristic evaluation with 16 UX researchers showed participants praised the system’s innovation but expressed concerns about future LLM Agent usage in UX studies.

Conclusion: UXAgent demonstrates the potential of LLM-simulated agents for usability testing study design iteration, though concerns remain about their broader application in UX research.

Abstract: Usability testing is a fundamental research method that user experience (UX) researchers use to evaluate and iterate their new designs. But what about evaluating and iterating the usability testing study design itself? Recent advances in Large Language Model-simulated Agent (LLM Agent) research inspired us to design UXAgent to support UX researchers in evaluating and iterating their study design before they conduct the real human-subject study. Our system features a Persona Generator module, an LLM Agent module, and a Universal Browser Connector module to automatically generate thousands of simulated users and to interactively test the target website. The system also provides a Result Viewer Interface so that the UX researchers can easily review and analyze the generated qualitative (e.g., agents’ post-study surveys) and quantitative data (e.g., agents’ interaction logs), or even interview agents directly. Through a heuristic evaluation with 16 UX researchers, participants praised the innovation of our system but also expressed concerns about the future of LLM Agent usage in UX studies.

Method: Systematically controlling training conditions (parameter sizes, optimization settings, random seeds) while keeping datasets identical, then analyzing the distinguishability of model outputs.

Result: LLM outputs remain distinguishable even when trained on exactly the same dataset, showing that training dynamics alone leave recognizable patterns (natural fingerprints).

Conclusion: Training dynamics systematically shape model behavior and should be explicitly considered in research on transparency, reliability, and interpretability.

Abstract: Recent studies have shown that the outputs from large language models (LLMs) can often reveal the identity of their source model. While this is a natural consequence of LLMs modeling the distribution of their training data, such identifiable traces may also reflect unintended characteristics with potential implications for fairness and misuse. In this work, we go one step further and show that even when LLMs are trained on exactly the same dataset, their outputs remain distinguishable, suggesting that training dynamics alone can leave recognizable patterns. We refer to these unintended, distinctive characteristics as natural fingerprints. By systematically controlling training conditions, we show that the natural fingerprints can emerge from subtle differences in the training process, such as parameter sizes, optimization settings, and even random seeds. These results suggest that training dynamics can systematically shape model behavior, independent of data or architecture, and should be explicitly considered in future research on transparency, reliability, and interpretability.

Method: Constructs CrossIC-PT samples by interleaving semantic-related bilingual Wikipedia documents into single context windows, using systematic segmentation and sliding window mechanisms to preserve coherence. Extends data availability through semantic retrieval from web-crawled corpus.

Result: Improves multilingual performance on three models (Llama-3.1-8B, Qwen2.5-7B, Qwen2.5-1.5B) across six languages with gains of 3.79%, 3.99%, and 1.95% respectively, with additional improvements after data augmentation.

Conclusion: CrossIC-PT provides an effective and scalable approach to enhance cross-lingual transfer in LLMs without heavy reliance on parallel resources, demonstrating significant performance improvements across diverse language models and target languages.

Abstract: Large language models (LLMs) exhibit remarkable multilingual capabilities despite English-dominated pre-training, attributed to cross-lingual mechanisms during pre-training. Existing methods for enhancing cross-lingual transfer remain constrained by parallel resources, suffering from limited linguistic and domain coverage. We propose Cross-lingual In-context Pre-training (CrossIC-PT), a simple and scalable approach that enhances cross-lingual transfer by leveraging semantically related bilingual texts via simple next-word prediction. We construct CrossIC-PT samples by interleaving semantic-related bilingual Wikipedia documents into a single context window. To access window size constraints, we implement a systematic segmentation policy to split long bilingual document pairs into chunks while adjusting the sliding window mechanism to preserve contextual coherence. We further extend data availability through a semantic retrieval framework to construct CrossIC-PT samples from web-crawled corpus. Experimental results demonstrate that CrossIC-PT improves multilingual performance on three models (Llama-3.1-8B, Qwen2.5-7B, and Qwen2.5-1.5B) across six target languages, yielding performance gains of 3.79%, 3.99%, and 1.95%, respectively, with additional improvements after data augmentation.

Method: Proposes AutoRefine, a reinforcement learning post-training framework with a “search-and-refine-during-think” paradigm. It incorporates explicit knowledge refinement steps between search calls and uses group relative policy optimization with tailored retrieval-specific rewards alongside answer correctness rewards.

Result: Experiments on single-hop and multi-hop QA benchmarks show AutoRefine significantly outperforms existing approaches, especially in complex multi-hop reasoning. The method issues frequent, higher-quality searches and effectively synthesizes evidence.

Conclusion: AutoRefine successfully addresses the limitations of noisy retrieval in retrieval-augmented reasoning by enabling iterative evidence refinement, demonstrating superior performance particularly in complex reasoning scenarios.

Abstract: Large language models have demonstrated impressive reasoning capabilities but are inherently limited by their knowledge reservoir. Retrieval-augmented reasoning mitigates this limitation by allowing LLMs to query external resources, but existing methods often retrieve irrelevant or noisy information, hindering accurate reasoning. In this paper, we propose AutoRefine, a reinforcement learning post-training framework that adopts a new “search-and-refine-during-think” paradigm. AutoRefine introduces explicit knowledge refinement steps between successive search calls, enabling the model to iteratively filter, distill, and organize evidence before generating an answer. Furthermore, we incorporate tailored retrieval-specific rewards alongside answer correctness rewards using group relative policy optimization. Experiments on single-hop and multi-hop QA benchmarks demonstrate that AutoRefine significantly outperforms existing approaches, particularly in complex, multi-hop reasoning scenarios. Detailed analysis shows that AutoRefine issues frequent, higher-quality searches and synthesizes evidence effectively.

Method: Controlled fine-tuning experiments across 132 translation directions to systematically test the effects of language diversity.

Result: Expanding language diversity improves translation quality for both unsupervised and supervised pairs, but benefits plateau or decrease beyond a threshold. Increased diversity creates more language-agnostic representations.

Conclusion: Language diversity in fine-tuning enhances translation performance by developing language-agnostic representations, though optimal diversity levels exist.

Abstract: Prior research diverges on language diversity in LLM fine-tuning: Some studies report benefits while others find no advantages. Through controlled fine-tuning experiments across 132 translation directions, we systematically resolve these disparities. We find that expanding language diversity during fine-tuning improves translation quality for both unsupervised and – surprisingly – supervised pairs, despite less diverse models being fine-tuned exclusively on these supervised pairs. However, benefits plateau or decrease beyond a certain diversity threshold. We show that increased language diversity creates more language-agnostic representations. These representational adaptations help explain the improved performance in models fine-tuned with greater diversity.

Method: Systematic evaluation of 6 LLMs (including 4 large reasoning models) with 5 pipelines and 2 types of formalism across 4 domains in few-shot settings

Result: LLM-as-formalizer underperforms LLM-as-solver; current LLMs fail to deliver promised accuracy, robustness, faithfulness, and efficiency due to limited ability to generate formal programs

Conclusion: Current LLMs are not yet effective as formalizers, and the paper provides detailed analysis and actionable remedies to improve LLM-as-formalizer approach

Abstract: A trending line of recent work advocates for using large language models (LLMs) as formalizers instead of as end-to-end solvers for logical reasoning problems. Instead of generating the solution, the LLM generates a formal program that derives a solution via an external solver. While performance gain of the seemingly scalable LLM-as-formalizer over the seemingly unscalable LLM-as-solver has been widely reported, we show that this superiority does not hold on real-life constraint satisfaction problems. On 4 domains, we systematically evaluate 6 LLMs including 4 large reasoning models with inference-time scaling, paired with 5 pipelines including 2 types of formalism. We show that in few-shot settings, LLM-as-formalizer underperforms LLM-as-solver. While LLM-as-formalizer promises accuracy, robustness, faithfulness, and efficiency, we observe that the present LLMs do not yet deliver any of those, as their limited ability to generate formal programs leads to failure to scale with complexity, hard-coded solutions, and excessive reasoning tokens. We present our detailed analysis and actionable remedies to drive future research that improves LLM-as-formalizer.

Method: Transform existing benchmarks into conversational tasks requiring effective communication in target languages, then measure LLMs’ task success rates without relying on language-specific tools or LLM-as-judge methods.

Result: MUG-Eval strongly correlates with established benchmarks (r > 0.75) and enables standardized comparisons across 30 languages spanning high, mid, and low-resource categories.

Conclusion: The framework provides a robust, resource-efficient solution for multilingual generation evaluation that can scale to thousands of languages.

Abstract: Evaluating text generation capabilities of large language models (LLMs) is challenging, particularly for low-resource languages where methods for direct assessment are scarce. We propose MUG-Eval, a novel framework that evaluates LLMs’ multilingual generation capabilities by transforming existing benchmarks into conversational tasks and measuring the LLMs’ accuracies on those tasks. We specifically designed these conversational tasks to require effective communication in the target language. Then, we simply use task success rate as a proxy for successful conversation generation. Our approach offers two key advantages: it is independent of language-specific NLP tools or annotated datasets, which are limited for most languages, and it does not rely on LLMs-as-judges, whose evaluation quality degrades outside a few high-resource languages. We evaluate 8 LLMs across 30 languages spanning high, mid, and low-resource categories, and we find that MUG-Eval correlates strongly with established benchmarks ($r$ > 0.75) while enabling standardized comparisons across languages and models. Our framework provides a robust and resource-efficient solution for evaluating multilingual generation that can be extended to thousands of languages.

Method: Developed CrPO, a modular alignment method that injects signals from multiple creativity dimensions into preference optimization. Used MuCE dataset with 200,000+ human responses and ratings from 30+ creativity assessments.

Result: CrPO-augmented models outperformed strong baselines including GPT-4o on both automated and human evaluations, producing more novel, diverse, and surprising generations while maintaining high output quality. Validated generalizability on NoveltyBench.

Conclusion: Directly optimizing for creativity within preference frameworks is a promising direction for advancing LLM creative capabilities without compromising output quality.

Abstract: While Large Language Models (LLMs) have demonstrated impressive performance across natural language generation tasks, their ability to generate truly creative content-characterized by novelty, diversity, surprise, and quality-remains limited. Existing methods for enhancing LLM creativity often focus narrowly on diversity or specific tasks, failing to address creativity’s multifaceted nature in a generalizable way. In this work, we propose Creative Preference Optimization (CrPO), a novel alignment method that injects signals from multiple creativity dimensions into the preference optimization objective in a modular fashion. We train and evaluate creativity-augmented versions of several models using CrPO and MuCE, a new large-scale human preference dataset spanning over 200,000 human-generated responses and ratings from more than 30 psychological creativity assessments. Our models outperform strong baselines, including GPT-4o, on both automated and human evaluations, producing more novel, diverse, and surprising generations while maintaining high output quality. Additional evaluations on NoveltyBench further confirm the generalizability of our approach. Together, our results demonstrate that directly optimizing for creativity within preference frameworks is a promising direction for advancing the creative capabilities of LLMs without compromising output quality.

Method: The study examines language mixing across 15 languages, 7 task difficulty levels, and 18 subject areas. They use constrained decoding to force models to reason in Latin or Han scripts and analyze how this affects performance. They also examine the alignment between reasoning traces and internal representations.

Result: The choice of reasoning language significantly affects performance - forcing models to reason in Latin or Han scripts via constrained decoding notably improves accuracy. The script composition of reasoning traces closely aligns with the model’s internal representations, indicating language mixing reflects latent processing preferences.

Conclusion: Language mixing reflects latent processing preferences in RLMs. The findings provide actionable insights for optimizing multilingual reasoning and open new directions for controlling reasoning languages to build more interpretable and adaptable RLMs.

Abstract: Reasoning language models (RLMs) excel at complex tasks by leveraging a chain-of-thought process to generate structured intermediate steps. However, language mixing, i.e., reasoning steps containing tokens from languages other than the prompt, has been observed in their outputs and shown to affect performance, though its impact remains debated. We present the first systematic study of language mixing in RLMs, examining its patterns, impact, and internal causes across 15 languages, 7 task difficulty levels, and 18 subject areas, and show how all three factors influence language mixing. Moreover, we demonstrate that the choice of reasoning language significantly affects performance: forcing models to reason in Latin or Han scripts via constrained decoding notably improves accuracy. Finally, we show that the script composition of reasoning traces closely aligns with that of the model’s internal representations, indicating that language mixing reflects latent processing preferences in RLMs. Our findings provide actionable insights for optimizing multilingual reasoning and open new directions for controlling reasoning languages to build more interpretable and adaptable RLMs.

Method: Created MemeSafetyBench with 50,430 instances pairing real meme images with harmful/benign instructions using a comprehensive safety taxonomy and LLM-based instruction generation. Evaluated multiple VLMs across single and multi-turn interactions.

Result: VLMs are more vulnerable to meme-based harmful prompts than synthetic or typographic images. Memes significantly increase harmful responses and decrease refusals compared to text-only inputs. Multi-turn interactions provide partial mitigation but elevated vulnerability persists.

Conclusion: The findings highlight the need for ecologically valid evaluations and stronger safety mechanisms. MemeSafetyBench is publicly available to facilitate better safety assessments.

Abstract: Rapid deployment of vision-language models (VLMs) magnifies safety risks, yet most evaluations rely on artificial images. This study asks: How safe are current VLMs when confronted with meme images that ordinary users share? To investigate this question, we introduce MemeSafetyBench, a 50,430-instance benchmark pairing real meme images with both harmful and benign instructions. Using a comprehensive safety taxonomy and LLM-based instruction generation, we assess multiple VLMs across single and multi-turn interactions. We investigate how real-world memes influence harmful outputs, the mitigating effects of conversational context, and the relationship between model scale and safety metrics. Our findings demonstrate that VLMs are more vulnerable to meme-based harmful prompts than to synthetic or typographic images. Memes significantly increase harmful responses and decrease refusals compared to text-only inputs. Though multi-turn interactions provide partial mitigation, elevated vulnerability persists. These results highlight the need for ecologically valid evaluations and stronger safety mechanisms. MemeSafetyBench is publicly available at https://github.com/oneonlee/Meme-Safety-Bench.

Method: MuseScorer integrates a Large Language Model with externally orchestrated retrieval: given a new idea, it retrieves semantically similar prior idea-buckets and zero-shot prompts the LLM to judge whether the idea fits an existing bucket or forms a new one.

Result: Across five datasets (N=1143 participants, n=16,294 ideas), MuseScorer matches human annotators in idea clustering structure (AMI = 0.59) and participant-level scoring (r = 0.89), with strong convergent and external validity.

Conclusion: The system enables scalable, intent-sensitive, and human-aligned originality assessment for creativity research, providing automated frequency-based originality scoring without human annotation.

Abstract: An objective, face-valid method for scoring idea originality is to measure each idea’s statistical infrequency within a population – an approach long used in creativity research. Yet, computing these frequencies requires manually bucketing idea rephrasings, a process that is subjective, labor-intensive, error-prone, and brittle at scale. We introduce MuseScorer, a fully automated, psychometrically validated system for frequency-based originality scoring. MuseScorer integrates a Large Language Model (LLM) with externally orchestrated retrieval: given a new idea, it retrieves semantically similar prior idea-buckets and zero-shot prompts the LLM to judge whether the idea fits an existing bucket or forms a new one. These buckets enable frequency-based originality scoring without human annotation. Across five datasets N_{participants}=1143, n_{ideas}=16,294), MuseScorer matches human annotators in idea clustering structure (AMI = 0.59) and participant-level scoring (r = 0.89), while demonstrating strong convergent and external validity. The system enables scalable, intent-sensitive, and human-aligned originality assessment for creativity research.

Method: Developed CLEAR framework with expert-curated labels and attribute-level comparison, examining presence/absence of conditions plus five key attributes: first occurrence, change, severity, descriptive location, and recommendation. Created CLEAR-Bench dataset with 100 chest X-ray reports annotated by board-certified radiologists.

Result: CLEAR achieves high accuracy in extracting clinical attributes and provides automated metrics that are strongly aligned with clinical judgment, enabling more comprehensive evaluation than prior methods.

Conclusion: CLEAR’s multi-dimensional, attribute-level outputs enable clinically interpretable radiology report evaluation that better captures nuanced clinical differences compared to existing metrics.

Abstract: Existing metrics often lack the granularity and interpretability to capture nuanced clinical differences between candidate and ground-truth radiology reports, resulting in suboptimal evaluation. We introduce a Clinically-grounded tabular framework with Expert-curated labels and Attribute-level comparison for Radiology report evaluation (CLEAR). CLEAR not only examines whether a report can accurately identify the presence or absence of medical conditions, but also assesses whether it can precisely describe each positively identified condition across five key attributes: first occurrence, change, severity, descriptive location, and recommendation. Compared to prior works, CLEAR’s multi-dimensional, attribute-level outputs enable a more comprehensive and clinically interpretable evaluation of report quality. Additionally, to measure the clinical alignment of CLEAR, we collaborate with five board-certified radiologists to develop CLEAR-Bench, a dataset of 100 chest X-ray reports from MIMIC-CXR, annotated across 6 curated attributes and 13 CheXpert conditions. Our experiments show that CLEAR achieves high accuracy in extracting clinical attributes and provides automated metrics that are strongly aligned with clinical judgment.

Method: Used lexical diversity and redundancy metrics to analyze synthetic prompts and responses generated by LLMs of different sizes with fine-grained and coarse persona descriptions, comparing against human-written prompts.

Result: Synthetic prompts are significantly less diverse than human-written ones. Persona prompting increases lexical diversity, especially in larger models, but fine-grained persona details provide minimal diversity gains compared to simple length specifications.

Conclusion: While persona prompting enhances diversity in LLM-generated text, particularly for larger models, the additional complexity of fine-grained persona descriptions offers little benefit over simpler length-based constraints.

Abstract: Fine-grained personas have recently been used for generating ‘diverse’ synthetic data for pre-training and supervised fine-tuning of Large Language Models (LLMs). In this work, we measure the diversity of persona-driven synthetically generated prompts and responses with a suite of lexical diversity and redundancy metrics. First, we find that synthetic prompts/instructions are significantly less diverse than human-written ones. Next, we sample responses from LLMs of different sizes with fine-grained and coarse persona descriptions to investigate how much fine-grained detail in persona descriptions contribute to generated text diversity. Our results indicate that persona prompting produces higher lexical diversity than prompting without personas, particularly in larger models. In contrast, adding fine-grained persona details yields minimal gains in diversity compared to simply specifying a length cutoff in the prompt.

Method: HydraRAG uses agent-driven exploration combining structured and unstructured retrieval, tri-factor cross-source verification (source trustworthiness assessment, cross-source corroboration, and entity-path alignment), and leverages graph structure to fuse heterogeneous sources and prune noise early.

Result: HydraRAG achieves state-of-the-art results on seven benchmark datasets, outperforming ToG-2 by an average of 20.3% (up to 30.1%) with GPT-3.5-Turbo, and enables smaller models like Llama-3.1-8B to achieve reasoning performance comparable to GPT-4-Turbo.

Conclusion: HydraRAG effectively addresses key limitations in hybrid RAG systems by unifying multiple information sources and verification mechanisms, demonstrating superior performance and enabling efficient reasoning in smaller language models.

Abstract: Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating external knowledge. Current hybrid RAG system retrieves evidence from both knowledge graphs (KGs) and text documents to support LLM reasoning. However, it faces challenges like handling multi-hop reasoning, multi-entity questions, multi-source verification, and effective graph utilization. To address these limitations, we present HydraRAG, a training-free framework that unifies graph topology, document semantics, and source reliability to support deep, faithful reasoning in LLMs. HydraRAG handles multi-hop and multi-entity problems through agent-driven exploration that combines structured and unstructured retrieval, increasing both diversity and precision of evidence. To tackle multi-source verification, HydraRAG uses a tri-factor cross-source verification (source trustworthiness assessment, cross-source corroboration, and entity-path alignment), to balance topic relevance with cross-modal agreement. By leveraging graph structure, HydraRAG fuses heterogeneous sources, guides efficient exploration, and prunes noise early. Comprehensive experiments on seven benchmark datasets show that HydraRAG achieves overall state-of-the-art results on all benchmarks with GPT-3.5-Turbo, outperforming the strong hybrid baseline ToG-2 by an average of 20.3% and up to 30.1%. Furthermore, HydraRAG enables smaller models (e.g., Llama-3.1-8B) to achieve reasoning performance comparable to that of GPT-4-Turbo. The source code is available on https://stevetantan.github.io/HydraRAG/.

Method: Uses pretrained NER for explicit targets, captures implicit targets via [CLS] tokens, computes attention-based relationships between targets and context, and injects relational vectors into final sentence representation.

Result: Achieves state-of-the-art performance, outperforming contrastive learning baselines by average 82.14%, faster convergence, and attention patterns align with human judgment.

Conclusion: AmpleHate effectively amplifies target-context relations for implicit hate detection, demonstrating superior performance, interpretability, and robustness compared to existing methods.

Abstract: Implicit hate speech detection is challenging due to its subtlety and reliance on contextual interpretation rather than explicit offensive words. Current approaches rely on contrastive learning, which are shown to be effective on distinguishing hate and non-hate sentences. Humans, however, detect implicit hate speech by first identifying specific targets within the text and subsequently interpreting how these target relate to their surrounding context. Motivated by this reasoning process, we propose AmpleHate, a novel approach designed to mirror human inference for implicit hate detection. AmpleHate identifies explicit target using a pretrained Named Entity Recognition model and capture implicit target information via [CLS] tokens. It computes attention-based relationships between explicit, implicit targets and sentence context and then, directly injects these relational vectors into the final sentence representation. This amplifies the critical signals of target-context relations for determining implicit hate. Experiments demonstrate that AmpleHate achieves state-of-the-art performance, outperforming contrastive learning baselines by an average of 82.14% and achieve faster convergence. Qualitative analyses further reveal that attention patterns produced by AmpleHate closely align with human judgement, underscoring its interpretability and robustness. Our code is publicly available at: https://github.com/leeyejin1231/AmpleHate.

Method: SEMMA uses LLMs to enrich relation identifiers, generates semantic embeddings to form a textual relation graph, and fuses it with the structural component.

Result: SEMMA outperforms structural baselines like ULTRA across 54 KGs, and is 2x more effective in challenging generalization settings with entirely unseen test-time relations.

Conclusion: Textual semantics are critical for generalization where structure alone fails, highlighting the need for foundation models that unify structural and linguistic signals.

Abstract: Knowledge Graph Foundation Models (KGFMs) have shown promise in enabling zero-shot reasoning over unseen graphs by learning transferable patterns. However, most existing KGFMs rely solely on graph structure, overlooking the rich semantic signals encoded in textual attributes. We introduce SEMMA, a dual-module KGFM that systematically integrates transferable textual semantics alongside structure. SEMMA leverages Large Language Models (LLMs) to enrich relation identifiers, generating semantic embeddings that subsequently form a textual relation graph, which is fused with the structural component. Across 54 diverse KGs, SEMMA outperforms purely structural baselines like ULTRA in fully inductive link prediction. Crucially, we show that in more challenging generalization settings, where the test-time relation vocabulary is entirely unseen, structural methods collapse while SEMMA is 2x more effective. Our findings demonstrate that textual semantics are critical for generalization in settings where structure alone fails, highlighting the need for foundation models that unify structural and linguistic signals in knowledge reasoning.

Method: CCPS applies targeted adversarial perturbations to final hidden states, extracts features reflecting the model’s response to these perturbations, and uses a lightweight classifier to predict answer correctness.

Result: CCPS reduces Expected Calibration Error by ~55% and Brier score by 21%, while increasing accuracy by 5 percentage points, AUPRC by 4 percentage points, and AUROC by 6 percentage points across four LLMs and three MMLU variants.

Conclusion: CCPS delivers an efficient, broadly applicable, and more accurate solution for estimating LLM confidence, thereby improving their trustworthiness.

Abstract: Miscalibration in Large Language Models (LLMs) undermines their reliability, highlighting the need for accurate confidence estimation. We introduce CCPS (Calibrating LLM Confidence by Probing Perturbed Representation Stability), a novel method analyzing internal representational stability in LLMs. CCPS applies targeted adversarial perturbations to final hidden states, extracts features reflecting the model’s response to these perturbations, and uses a lightweight classifier to predict answer correctness. CCPS was evaluated on LLMs from 8B to 32B parameters (covering Llama, Qwen, and Mistral architectures) using MMLU and MMLU-Pro benchmarks in both multiple-choice and open-ended formats. Our results show that CCPS significantly outperforms current approaches. Across four LLMs and three MMLU variants, CCPS reduces Expected Calibration Error by approximately 55% and Brier score by 21%, while increasing accuracy by 5 percentage points, Area Under the Precision-Recall Curve by 4 percentage points, and Area Under the Receiver Operating Characteristic Curve by 6 percentage points, all relative to the strongest prior method. CCPS delivers an efficient, broadly applicable, and more accurate solution for estimating LLM confidence, thereby improving their trustworthiness.

Method: Leverages multiple state-of-the-art LLMs to generate multilingual dialogues from varied seed contexts, uses GPT-4.1 for multidimensional analysis, and creates a human-annotated meta-evaluation benchmark.

Result: Uncovered noticeable cross-lingual performance differences and found that state-of-the-art judges fail to reliably detect nuanced issues in chatbot responses.

Conclusion: MEDAL provides a more representative evaluation framework that reveals critical limitations in current LLM-based chatbot evaluation methods.

Abstract: Evaluating the quality of open-domain chatbots has become increasingly reliant on LLMs acting as automatic judges. However, existing meta-evaluation benchmarks are static, outdated, and lacking in multilingual coverage, limiting their ability to fully capture subtle weaknesses in evaluation. We introduce MEDAL, an automated multi-agent framework for curating more representative and diverse open-domain dialogue evaluation benchmarks. Our approach leverages several state-of-the-art LLMs to generate user-chatbot multilingual dialogues, conditioned on varied seed contexts. Then, a strong LLM (GPT-4.1) is used for a multidimensional analysis of the performance of the chatbots, uncovering noticeable cross-lingual performance differences. Guided by this large-scale evaluation, we curate a new meta-evaluation multilingual benchmark and human-annotate samples with nuanced quality judgments. This benchmark is then used to assess the ability of several reasoning and non-reasoning LLMs to act as evaluators of open-domain dialogues. Using MEDAL, we uncover that state-of-the-art judges fail to reliably detect nuanced issues such as lack of empathy, commonsense, or relevance.

Method: Proposes Budget EAGLE (Beagle) - a cross-attention-based Transformer decoder that eliminates pooling/auxiliary components. Uses Two-Stage Block-Attention Training for stable training in block-level attention scenarios.

Result: Beagle achieves competitive inference speedups and higher training efficiency than EAGLE-v2 across multiple LLMs and datasets, with stable memory usage during training.

Conclusion: Beagle offers a strong alternative architecture for speculative decoding, providing simplified design, better training efficiency, and comparable performance to state-of-the-art methods.

Abstract: Speculative decoding (SD) is a widely adopted approach for accelerating inference in large language models (LLMs), particularly when the draft and target models are well aligned. However, state-of-the-art SD methods typically rely on tightly coupled, self-attention-based Transformer decoders, often augmented with auxiliary pooling or fusion layers. This coupling makes them increasingly complex and harder to generalize across different models. We present Budget EAGLE (Beagle), the first, to our knowledge, cross-attention-based Transformer decoder SD model that achieves performance on par with leading self-attention SD models (EAGLE-v2) while eliminating the need for pooling or auxiliary components, simplifying the architecture, improving training efficiency, and maintaining stable memory usage during training-time simulation. To enable effective training of this novel architecture, we propose Two-Stage Block-Attention Training, a new method that achieves training stability and convergence efficiency in block-level attention scenarios. Extensive experiments across multiple LLMs and datasets show that Beagle achieves competitive inference speedups and higher training efficiency than EAGLE-v2, offering a strong alternative for architectures in speculative decoding.

Method: Introduce a language-agnostic benchmark for in-context learning, analyze forgetting patterns, and propose curriculum learning and exponential moving average of weights as alternatives to mitigate English dependency.

Result: English inclusion prevents catastrophic forgetting during early CPT stages, which is critical for downstream task performance. Without English, models suffer parameter shifts that damage generalization even when validation perplexity appears unaffected.

Conclusion: The study reveals the dynamics of emergent abilities in language adaptation CPT and provides foundation for designing more effective methods, with curriculum learning and EMA as viable alternatives to English inclusion.

Abstract: Continued pretraining (CPT) is a popular approach to adapt existing large language models (LLMs) to new languages. When doing so, it is common practice to include a portion of English data in the mixture, but its role has not been carefully studied to date. In this work, we show that including English does not impact validation perplexity, yet it is critical for the emergence of downstream capabilities in the target language. We introduce a language-agnostic benchmark for in-context learning (ICL), which reveals catastrophic forgetting early on CPT when English is not included. This in turn damages the ability of the model to generalize to downstream prompts in the target language as measured by perplexity, even if it does not manifest in terms of accuracy until later in training, and can be tied to a big shift in the model parameters. Based on these insights, we introduce curriculum learning and exponential moving average (EMA) of weights as effective alternatives to mitigate the need for English. All in all, our work sheds light into the dynamics by which emergent abilities arise when doing CPT for language adaptation, and can serve as a foundation to design more effective methods in the future.

Method: Study how DSL and PPI performance scales with number of expert annotations, comparing them across various tasks to analyze bias reduction and empirical efficiency in finite samples.

Result: Both methods achieve low bias with large datasets, but DSL often outperforms PPI on bias reduction and efficiency, though with less consistency across different datasets.

Conclusion: There is a bias-variance tradeoff at the debiasing method level, indicating need for better metrics to quantify efficiency in finite samples.

Abstract: Large language models (LLMs) offer an inexpensive yet powerful way to annotate text, but are often inconsistent when compared with experts. These errors can bias downstream estimates of population parameters such as regression coefficients and causal effects. To mitigate this bias, researchers have developed debiasing methods such as Design-based Supervised Learning (DSL) and Prediction-Powered Inference (PPI), which promise valid estimation by combining LLM annotations with a limited number of expensive expert annotations. Although these methods produce consistent estimates under theoretical assumptions, it is unknown how they compare in finite samples of sizes encountered in applied research. We make two contributions. First, we study how each methods performance scales with the number of expert annotations, highlighting regimes where LLM bias or limited expert labels significantly affect results. Second, we compare DSL and PPI across a range of tasks, finding that although both achieve low bias with large datasets, DSL often outperforms PPI on bias reduction and empirical efficiency, but its performance is less consistent across datasets. Our findings indicate that there is a bias-variance tradeoff at the level of debiasing methods, calling for more research on developing metrics for quantifying their efficiency in finite samples.

Method: Constructed FineHarm dataset with 29K fine-grained annotations, and proposed SCM trained with dual supervision (response- and token-level labels) to monitor LLM output streams in real-time.

Result: SCM achieves 0.95+ macro F1 score comparable to full detection, while only processing first 18% of tokens on average. Also improves safety alignment as pseudo-harmfulness annotator.

Conclusion: The proposed streaming content monitor effectively bridges the training-inference gap in partial detection and provides timely harmful content moderation with minimal latency.

Abstract: Though safety alignment has been applied to most large language models (LLMs), LLM service providers generally deploy a subsequent moderation as the external safety guardrail in real-world products. Existing moderators mainly practice a conventional full detection, which determines the harmfulness based on the complete LLM output, causing high service latency. Recent works pay more attention to partial detection where moderators oversee the generation midway and early stop the output if harmfulness is detected, but they directly apply moderators trained with the full detection paradigm to incomplete outputs, introducing a training-inference gap that lowers the performance. In this paper, we explore how to form a data-and-model solution that natively supports partial detection. For the data, we construct FineHarm, a dataset consisting of 29K prompt-response pairs with fine-grained annotations to provide reasonable supervision for token-level training. Then, we propose the streaming content monitor, which is trained with dual supervision of response- and token-level labels and can follow the output stream of LLM to make a timely judgment of harmfulness. Experiments show that SCM gains 0.95+ in macro F1 score that is comparable to full detection, by only seeing the first 18% of tokens in responses on average. Moreover, the SCM can serve as a pseudo-harmfulness annotator for improving safety alignment and lead to a higher harmlessness score than DPO.

Method: Evaluated performance of generation strategies (demonstrations, label-based summaries, self-revision) and their combinations across 11 diverse languages using 3 NLP tasks and 4 open-source LLMs.

Result: Strategic combinations, particularly target-language demonstrations with LLM-based revisions, yield strong performance. Smart prompting can reduce the advantage of larger LLMs.

Conclusion: Efficient generation strategies exist for synthetic data generation in low-resource scenarios using smaller models, with performance gaps as small as 5% compared to real data.

Abstract: Large Language Models (LLMs) are increasingly used to generate synthetic textual data for training smaller specialized models. However, a comparison of various generation strategies for low-resource language settings is lacking. While various prompting strategies have been proposed, such as demonstrations, label-based summaries, and self-revision, their comparative effectiveness remains unclear, especially for low-resource languages. In this paper, we systematically evaluate the performance of these generation strategies and their combinations across 11 typologically diverse languages, including several extremely low-resource ones. Using three NLP tasks and four open-source LLMs, we assess downstream model performance on generated versus gold-standard data. Our results show that strategic combinations of generation methods, particularly target-language demonstrations with LLM-based revisions, yield strong performance, narrowing the gap with real data to as little as 5% in some settings. We also find that smart prompting techniques can reduce the advantage of larger LLMs, highlighting efficient generation strategies for synthetic data generation in low-resource scenarios with smaller models.

Method: Proposes Information Gain-based Decisiveness-aware Token handling (IGD) strategy that quantifies token decisiveness using Information Gain. IGD downweights low-IG tokens during tuning and rebalances decoding to emphasize high-IG tokens.

Result: Extensive experiments on four benchmark datasets with two LLM backbones show IGD consistently improves recommendation accuracy, achieving significant gains on widely used ranking metrics compared to strong baselines.

Conclusion: IGD effectively prioritizes high-decisiveness tokens in LLM-based recommendation, demonstrating that moving beyond pure likelihood maximization through token-level decisiveness awareness leads to better recommendation performance.

Abstract: Large Language Models (LLMs) have shown strong potential for recommendation by framing item prediction as a token-by-token language generation task. However, existing methods treat all item tokens equally, simply pursuing likelihood maximization during both optimization and decoding. This overlooks crucial token-level differences in decisiveness-many tokens contribute little to item discrimination yet can dominate optimization or decoding. To quantify token decisiveness, we propose a novel perspective that models item generation as a decision process, measuring token decisiveness by the Information Gain (IG) each token provides in reducing uncertainty about the generated item. Our empirical analysis reveals that most tokens have low IG but often correspond to high logits, disproportionately influencing training loss and decoding, which may impair model performance. Building on these insights, we introduce an Information Gain-based Decisiveness-aware Token handling (IGD) strategy that integrates token decisiveness into both tuning and decoding. Specifically, IGD downweights low-IG tokens during tuning and rebalances decoding to emphasize tokens with high IG. In this way, IGD moves beyond pure likelihood maximization, effectively prioritizing high-decisiveness tokens. Extensive experiments on four benchmark datasets with two LLM backbones demonstrate that IGD consistently improves recommendation accuracy, achieving significant gains on widely used ranking metrics compared to strong baselines.

Method: Comprehensive study analyzing the impact of ASR transcription errors on speaker attribution performance, examining how transcription error types and ASR system properties affect attribution accuracy.

Result: Speaker attribution is resilient to word-level transcription errors, and recovering true transcripts is minimally correlated with attribution performance. ASR-based attribution performs as well or better than human-transcribed data.

Conclusion: ASR transcription errors can capture speaker-specific features that reveal speaker identity, making speaker attribution from errorful ASR transcripts a viable and effective approach in real-world applications.

Abstract: Speaker attribution from speech transcripts is the task of identifying a speaker from the transcript of their speech based on patterns in their language use. This task is especially useful when the audio is unavailable (e.g. deleted) or unreliable (e.g. anonymized speech). Prior work in this area has primarily focused on the feasibility of attributing speakers using transcripts produced by human annotators. However, in real-world settings, one often only has more errorful transcripts produced by automatic speech recognition (ASR) systems. In this paper, we conduct what is, to our knowledge, the first comprehensive study of the impact of automatic transcription on speaker attribution performance. In particular, we study the extent to which speaker attribution performance degrades in the face of transcription errors, as well as how properties of the ASR system impact attribution. We find that attribution is surprisingly resilient to word-level transcription errors and that the objective of recovering the true transcript is minimally correlated with attribution performance. Overall, our findings suggest that speaker attribution on more errorful transcripts produced by ASR is as good, if not better, than attribution based on human-transcribed data, possibly because ASR transcription errors can capture speaker-specific features revealing of speaker identity.

Method: Propose translationese-index (T-index) computed from likelihood ratios of two contrastively fine-tuned language models. Use synthesized translations and real translations to evaluate generalizability and validity against human judgments.

Result: T-index generalizes to unseen genres, authors, and language pairs. It effectively captures translationese with alignment to human ratings using only 1-5k synthetic data pairs with 0.5B LMs. Low correlation with BLEU and COMET suggests it’s complementary to existing MT QE metrics.

Conclusion: T-index provides a valid graded measure of translationese that generalizes well and serves as a complementary metric in machine translation quality estimation.

Abstract: Translationese refers to linguistic properties that usually occur in translated texts. Previous works study translationese by framing it as a binary classification between original texts and translated texts. In this paper, we argue that translationese should be graded instead of binary and propose the first measure for translationese – the translationese-index (T-index), computed from the likelihood ratios of two contrastively fine-tuned language models (LMs). We use synthesized translations and translations in the wild to evaluate T-index’s generalizability in cross-domain settings and its validity against human judgments. Our results show that T-index can generalize to unseen genres, authors, and language pairs. Moreover, T-index computed using two 0.5B LMs fine-tuned on only 1-5k pairs of synthetic data can effectively capture translationese, as demonstrated by alignment with human pointwise ratings and pairwise judgments. Additionally, the correlation between T-index and existing machine translation (MT) quality estimation (QE) metrics such as BLEU and COMET is low, suggesting that T-index is not covered by these metrics and can serve as a complementary metric in MT QE.

Method: 1) Use a lightweight BERT model to pre-encode input text into a single Contextual token prepended to the LLM input. 2) Concatenate last hidden states of Contextual and EOS tokens as final embedding to mitigate recency bias from last-token pooling.

Result: Achieves state-of-the-art performance on MTEB among models trained only on public retrieval datasets, while reducing sequence length by up to 85% and inference time by up to 82% compared to best-performing methods.

Conclusion: Causal2Vec provides an effective and efficient approach for leveraging decoder-only LLMs as embedding models without architectural changes or significant computational overhead, demonstrating superior performance and efficiency.

Abstract: Decoder-only large language models (LLMs) are increasingly used to build embedding models that effectively encode the semantic information of natural language texts into dense vector representations for various embedding tasks. However, many existing methods primarily focus on removing the causal attention mask in LLMs to enable bidirectional attention, potentially undermining the model’s ability to extract semantic information acquired during pretraining. Additionally, leading unidirectional approaches often rely on extra input text to overcome the inherent limitations of causal attention, inevitably increasing computational costs. In this work, we propose Causal2Vec, a general-purpose embedding model tailored to enhance the performance of decoder-only LLMs without altering their original architectures or introducing significant computational overhead. Specifically, we first employ a lightweight BERT-style model to pre-encode the input text into a single Contextual token, which is then prepended to the LLM’s input sequence, allowing each token to capture contextualized information even without attending to future tokens. Furthermore, to mitigate the recency bias introduced by last-token pooling and help LLMs better leverage the semantic information encoded in the Contextual token, we concatenate the last hidden states of Contextual and EOS tokens as the final text embedding. In practice, Causal2Vec achieves state-of-the-art performance on the Massive Text Embeddings Benchmark (MTEB) among models trained solely on publicly available retrieval datasets, while reducing the required sequence length by up to 85% and inference time by up to 82% compared to best-performing methods.

Method: XAutoLM uses meta-learning to extract task- and system-level meta-features from past successes/failures, biasing sampling toward valuable configurations and away from costly dead ends.

Result: On 6 benchmarks, XAutoLM surpassed zero-shot optimizer’s peak F1 on 5 tasks, reduced evaluation time by 4.5x, cut search error ratios by 7x, and found 50% more pipelines above the Pareto front.

Conclusion: XAutoLM enables resource-efficient Green AI fine-tuning, outperforming simpler memory-based approaches that suffer from negative transfer.

Abstract: Experts in machine learning leverage domain knowledge to navigate decisions in model selection, hyperparameter optimization, and resource allocation. This is particularly critical for fine-tuning language models (LMs), where repeated trials incur substantial computational overhead and environmental impact. However, no existing automated framework simultaneously tackles the entire model selection and hyperparameter optimization (HPO) task for resource-efficient LM fine-tuning. We introduce XAutoLM, a meta-learning-augmented AutoML framework that reuses past experiences to optimize discriminative and generative LM fine-tuning pipelines efficiently. XAutoLM learns from stored successes and failures by extracting task- and system-level meta-features to bias its sampling toward valuable configurations and away from costly dead ends. On four text classification and two question-answering benchmarks, XAutoLM surpasses zero-shot optimizer’s peak F1 on five of six tasks, cuts mean evaluation time of pipelines by up to 4.5x, reduces search error ratios by up to sevenfold, and uncovers up to 50% more pipelines above the zero-shot Pareto front. In contrast, simpler memory-based baselines suffer negative transfer. We release XAutoLM and our experience store to catalyze resource-efficient, Green AI fine-tuning in the NLP community.

Method: Created a human-authored Thai dataset through multi-stage quality control, covering four professional domains and seven task types. Conducted zero-shot evaluation and instruction tuning studies with ablations to isolate native supervision effects.

Result: Models fine-tuned on WangchanThaiInstruct outperformed those using translated data in both in-domain and out-of-domain benchmarks, showing significant performance gaps on culturally and professionally specific tasks.

Conclusion: Culturally and professionally grounded instruction data is essential for improving LLM alignment in low-resource, linguistically diverse settings.

Abstract: Large language models excel at instruction-following in English, but their performance in low-resource languages like Thai remains underexplored. Existing benchmarks often rely on translations, missing cultural and domain-specific nuances needed for real-world use. We present WangchanThaiInstruct, a human-authored Thai dataset for evaluation and instruction tuning, covering four professional domains and seven task types. Created through a multi-stage quality control process with annotators, domain experts, and AI researchers, WangchanThaiInstruct supports two studies: (1) a zero-shot evaluation showing performance gaps on culturally and professionally specific tasks, and (2) an instruction tuning study with ablations isolating the effect of native supervision. Models fine-tuned on WangchanThaiInstruct outperform those using translated data in both in-domain and out-of-domain benchmarks. These findings underscore the need for culturally and professionally grounded instruction data to improve LLM alignment in low-resource, linguistically diverse settings.

Method: Introduces HeTLM (Heterogeneity aware Training of Language Model), a clusterwise LM training approach using page-level tokenizers and heterogeneous cluster-specific parameters to capture subjective user behaviors.

Result: Small LMs with page-level tokenizers outperform large pretrained/finetuned LMs; HeTLM with cluster-specific parameters outperforms single LMs of same parameter count; achieves higher mean and lower variance in generation performance.

Conclusion: Clusterwise training with small LMs and appropriate tokenization better captures subjective user behaviors than large single LMs, leading to improved alignment through reduced performance variance across users.

Abstract: A Large Language Model (LLM) offers versatility across domains and tasks, purportedly benefiting users with a wide variety of behaviors and preferences. We question this perception about an LLM when users have inherently subjective behaviors and preferences, as seen in their ubiquitous and idiosyncratic browsing of websites or apps. The sequential behavior logs of pages, thus generated, form something akin to each user’s self-constructed “language”, albeit without the structure and grammar imbued in natural languages. We ask: (i) Can a small LM represent the “language of browsing” better than a large LM? (ii) Can an LM with a single set of parameters (or, single LM) adequately capture myriad users’ heterogeneous, subjective behaviors and preferences? (iii) Can a single LM with high average performance, yield low variance in performance to make alignment good at user level? We introduce clusterwise LM training, HeTLM (Heterogeneity aware Training of Language Model), appropriate for subjective behaviors. We find that (i) a small LM trained using a page-level tokenizer outperforms large pretrained or finetuned LMs; (ii) HeTLM with heterogeneous cluster specific set of parameters outperforms a single LM of the same family, controlling for the number of parameters; and (iii) a higher mean and a lower variance in generation ensues, implying improved alignment.

Method: Introduced OpenWHO corpus with 2,978 documents and 26,824 sentences from WHO’s e-learning platform, spanning over 20 languages (9 low-resource). Evaluated modern LLMs against traditional MT models and investigated LLM context utilization effects.

Result: LLMs consistently outperformed traditional MT models, with Gemini 2.5 Flash achieving +4.79 ChrF point improvement over NLLB-54B on low-resource test set. Document-level translation benefits were most pronounced in specialized health domains.

Conclusion: The OpenWHO corpus is released to encourage further research into low-resource MT in health domain, demonstrating LLMs’ superiority over traditional approaches in specialized translation tasks.

Abstract: In machine translation (MT), health is a high-stakes domain characterised by widespread deployment and domain-specific vocabulary. However, there is a lack of MT evaluation datasets for low-resource languages in this domain. To address this gap, we introduce OpenWHO, a document-level parallel corpus of 2,978 documents and 26,824 sentences from the World Health Organization’s e-learning platform. Sourced from expert-authored, professionally translated materials shielded from web-crawling, OpenWHO spans a diverse range of over 20 languages, of which nine are low-resource. Leveraging this new resource, we evaluate modern large language models (LLMs) against traditional MT models. Our findings reveal that LLMs consistently outperform traditional MT models, with Gemini 2.5 Flash achieving a +4.79 ChrF point improvement over NLLB-54B on our low-resource test set. Further, we investigate how LLM context utilisation affects accuracy, finding that the benefits of document-level translation are most pronounced in specialised domains like health. We release the OpenWHO corpus to encourage further research into low-resource MT in the health domain.

Method: Uses reinforcement learning to optimize compression without predefined labels, generating summaries that maximize LLM answer accuracy through end-task performance optimization.

Result: Achieves 3% compression ratio while improving average Exact Match score by 3.3 points across four datasets, avoiding performance degradation compared to full document prepending.

Conclusion: CORE enables effective lossless context compression for RAG systems through reinforcement learning optimization, significantly reducing computational costs while enhancing factual accuracy.

Abstract: Retrieval-Augmented Generation (RAG) has emerged as a promising approach to enhance the timeliness of knowledge and the factual accuracy of responses in Large Language Models (LLMs). However, the inclusion of excessive retrieved documents substantially increases the input length, leading to higher computational costs. Previous studies have attempted to compress retrieved documents into shorter texts before in-context integration, but such methods often compromise end-task performance. The lack of well-defined compression targets forces many approaches to rely on fixed heuristics, which cannot guarantee that the compressed content will effectively support the end task. To address these limitations, we propose CORE, a novel method designed to achieve lossless context compression for RAG. CORE employs reinforcement learning to optimize the compression process without relying on predefined compression labels, which enables the compressor to generate summaries that maximize the accuracy of answers generated by the LLM. Extensive experiments on four datasets demonstrate the superiority of our approach. With a high compression ratio of 3%, our method not only avoids performance degradation compared to prepending full documents across all datasets but also improves the average Exact Match (EM) score by 3.3 points. The code will be released soon.

Method: Introduces DCSRM that decomposes word embeddings from large language models into multiple sub-embeddings, identifies interpretable semantic subdimensions, and uses voxel-wise encoding models to map these subdimensions to brain activation patterns.

Result: The framework successfully identifies interpretable semantic subdimensions, reveals that semantic dimensions are structured by distinct principles with polarity as a key decomposition factor, and demonstrates neural correlates supporting cognitive plausibility.

Conclusion: The work provides more fine-grained interpretable semantic subdimensions for conceptual meaning and validates their neuroscientific plausibility through brain mapping, advancing our understanding of semantic organization.

Abstract: Understanding the core dimensions of conceptual semantics is fundamental to uncovering how meaning is organized in language and the brain. Existing approaches often rely on predefined semantic dimensions that offer only broad representations, overlooking finer conceptual distinctions. This paper proposes a novel framework to investigate the subdimensions underlying coarse-grained semantic dimensions. Specifically, we introduce a Disentangled Continuous Semantic Representation Model (DCSRM) that decomposes word embeddings from large language models into multiple sub-embeddings, each encoding specific semantic information. Using these sub-embeddings, we identify a set of interpretable semantic subdimensions. To assess their neural plausibility, we apply voxel-wise encoding models to map these subdimensions to brain activation. Our work offers more fine-grained interpretable semantic subdimensions of conceptual meaning. Further analyses reveal that semantic dimensions are structured according to distinct principles, with polarity emerging as a key factor driving their decomposition into subdimensions. The neural correlates of the identified subdimensions support their cognitive and neuroscientific plausibility.

Method: Middo establishes a closed-loop optimization system with: (1) self-referential diagnostic module using tri-axial model signals (loss patterns, embedding cluster dynamics, self-alignment scores), (2) adaptive optimization engine that transforms suboptimal samples while preserving semantic integrity, and (3) dynamic learning principles for continuous evolution with model capability.

Result: Experiments on multiple benchmarks show Middo consistently enhances seed data quality and boosts LLM performance with 7.15% average accuracy improvement while maintaining original dataset scale.

Conclusion: This work establishes a new paradigm for sustainable LLM training through dynamic human-AI co-evolution of data and models.

Abstract: Supervised Fine-Tuning (SFT) Large Language Models (LLM) fundamentally rely on high-quality training data. While data selection and data synthesis are two common strategies to improve data quality, existing approaches often face limitations in static dataset curation that fail to adapt to evolving model capabilities. In this paper, we introduce Middo, a self-evolving Model-informed dynamic data optimization framework that uses model-aware data selection and context-preserving data refinement. Unlike conventional one-off filtering/synthesis methods, our framework establishes a closed-loop optimization system: (1) A self-referential diagnostic module proactively identifies suboptimal samples through tri-axial model signals - loss patterns (complexity), embedding cluster dynamics (diversity), and self-alignment scores (quality); (2) An adaptive optimization engine then transforms suboptimal samples into pedagogically valuable training points while preserving semantic integrity; (3) This optimization process continuously evolves with model capability through dynamic learning principles. Experiments on multiple benchmarks demonstrate that our Middo consistently enhances the quality of seed data and boosts LLM’s performance with improving accuracy by 7.15% on average while maintaining the original dataset scale. This work establishes a new paradigm for sustainable LLM training through dynamic human-AI co-evolution of data and models. Our datasets, models, and code are coming soon. Our datasets, models, and code are publicly available at https://github.com/Word2VecT/Middo.

Method: Core Parameter Isolation Fine-Tuning: independently fine-tune on each task to identify core regions, group tasks by region overlap, transplant core parameters into unified backbone, integrate non-core parameters via SLERP, and use pipelined SFT with frozen core regions.

Result: Extensive experiments show significant alleviation of task interference and forgetting, consistently outperforming vanilla multi-task and multi-stage fine-tuning baselines on multiple public benchmarks.

Conclusion: CPI-FT effectively addresses the seesaw phenomenon in multi-task fine-tuning through core parameter isolation and fusion techniques, providing a robust framework for adapting LLMs to multiple downstream tasks.

Abstract: Supervised fine-tuning (SFT) is a pivotal approach to adapting large language models (LLMs) for downstream tasks; however, performance often suffers from the ``seesaw phenomenon’’, where indiscriminate parameter updates yield progress on certain tasks at the expense of others. To address this challenge, we propose a novel \emph{Core Parameter Isolation Fine-Tuning} (CPI-FT) framework. Specifically, we first independently fine-tune the LLM on each task to identify its core parameter regions by quantifying parameter update magnitudes. Tasks with similar core regions are then grouped based on region overlap, forming clusters for joint modeling. We further introduce a parameter fusion technique: for each task, core parameters from its individually fine-tuned model are directly transplanted into a unified backbone, while non-core parameters from different tasks are smoothly integrated via Spherical Linear Interpolation (SLERP), mitigating destructive interference. A lightweight, pipelined SFT training phase using mixed-task data is subsequently employed, while freezing core regions from prior tasks to prevent catastrophic forgetting. Extensive experiments on multiple public benchmarks demonstrate that our approach significantly alleviates task interference and forgetting, consistently outperforming vanilla multi-task and multi-stage fine-tuning baselines.

Method: Combines UMLS and LLM-KB knowledge with structured prompts (multilingual variants, medical synonyms, UMLS definitions) and LoRA-based fine-tuning on a 2,999-sentence parallel corpus, evaluated on four open-source LLMs.

Result: Significant translation quality improvements across all models, with Phi-4 achieving BLEU 44.23, chrF++ 28.91, and COMET 0.863, outperforming GPT-4o and GPT-4o-mini baselines.

Conclusion: Structured knowledge integration through MedCOD effectively enhances LLM performance for medical translation, with both prompting and adaptation contributing independently to gains.

Abstract: We present MedCOD (Medical Chain-of-Dictionary), a hybrid framework designed to improve English-to-Spanish medical translation by integrating domain-specific structured knowledge into large language models (LLMs). MedCOD integrates domain-specific knowledge from both the Unified Medical Language System (UMLS) and the LLM-as-Knowledge-Base (LLM-KB) paradigm to enhance structured prompting and fine-tuning. We constructed a parallel corpus of 2,999 English-Spanish MedlinePlus articles and a 100-sentence test set annotated with structured medical contexts. Four open-source LLMs (Phi-4, Qwen2.5-14B, Qwen2.5-7B, and LLaMA-3.1-8B) were evaluated using structured prompts that incorporated multilingual variants, medical synonyms, and UMLS-derived definitions, combined with LoRA-based fine-tuning. Experimental results demonstrate that MedCOD significantly improves translation quality across all models. For example, Phi-4 with MedCOD and fine-tuning achieved BLEU 44.23, chrF++ 28.91, and COMET 0.863, surpassing strong baseline models like GPT-4o and GPT-4o-mini. Ablation studies confirm that both MedCOD prompting and model adaptation independently contribute to performance gains, with their combination yielding the highest improvements. These findings highlight the potential of structured knowledge integration to enhance LLMs for medical translation tasks.

Method: Uses two novel designs: Zero-computation Experts for dynamic computational budget allocation (activating 18.6B-31.3B parameters per token) and Shortcut-connected MoE to improve computation-communication overlap. Implements comprehensive scaling framework with hyperparameter transfer, model-growth initialization, stability suite, and deterministic computation.

Result: Trained on 20+ trillion tokens in 30 days, achieves over 100 tokens per second inference at $0.70 per million output tokens. Delivers highly competitive performance among leading models with exceptional strengths in agentic tasks.

Conclusion: LongCat-Flash successfully demonstrates scalable efficiency through innovative architectural designs and achieves state-of-the-art performance while being open-sourced to foster community research.

Abstract: We introduce LongCat-Flash, a 560-billion-parameter Mixture-of-Experts (MoE) language model designed for both computational efficiency and advanced agentic capabilities. Stemming from the need for scalable efficiency, LongCat-Flash adopts two novel designs: (a) Zero-computation Experts, which enables dynamic computational budget allocation and activates 18.6B-31.3B (27B on average) per token depending on contextual demands, optimizing resource usage. (b) Shortcut-connected MoE, which enlarges the computation-communication overlap window, demonstrating notable gains in inference efficiency and throughput compared to models of a comparable scale. We develop a comprehensive scaling framework for large models that combines hyperparameter transfer, model-growth initialization, a multi-pronged stability suite, and deterministic computation to achieve stable and reproducible training. Notably, leveraging the synergy among scalable architectural design and infrastructure efforts, we complete model training on more than 20 trillion tokens within 30 days, while achieving over 100 tokens per second (TPS) for inference at a cost of $0.70 per million output tokens. To cultivate LongCat-Flash towards agentic intelligence, we conduct a large-scale pre-training on optimized mixtures, followed by targeted mid- and post-training on reasoning, code, and instructions, with further augmentation from synthetic data and tool use tasks. Comprehensive evaluations demonstrate that, as a non-thinking foundation model, LongCat-Flash delivers highly competitive performance among other leading models, with exceptional strengths in agentic tasks. The model checkpoint of LongCat-Flash is open-sourced to foster community research. LongCat Chat: https://longcat.ai Hugging Face: https://huggingface.co/meituan-longcat GitHub: https://github.com/meituan-longcat

Method: The study examines RALM calibration across different knowledge states, tests Refusal-aware Instruction Tuning and In-Context Fine-tuning methods, and develops a simple refusal method for post-trained models.

Result: LLMs show significant over-refusal behavior. In-context fine-tuning mitigates over-refusal while R-tuning magnifies it, but refusal ability may conflict with answer quality. A simple refusal method improves overall answer quality.

Conclusion: The study provides comprehensive understanding of factors influencing RALM systems, highlighting the over-refusal problem and offering solutions to balance refusal capability with answer quality.

Abstract: Existing Large Language Models (LLMs) occasionally generate plausible yet factually incorrect responses, known as hallucinations. Researchers are primarily using two approaches to mitigate hallucinations, namely Retrieval Augmented Language Models (RALMs) and refusal post-training. However, current research predominantly emphasizes their individual effectiveness while overlooking the evaluation of the refusal capability of RALMs. In this study, we ask the fundamental question: Do RALMs know when they don’t know? Specifically, we ask three questions. First, are RALMs well-calibrated regarding different internal and external knowledge states? We examine the influence of various factors. Contrary to expectations, we find that LLMs exhibit significant \textbf{over-refusal} behavior. Then, how does refusal post-training affect the over-refusal issue? We investigate the Refusal-aware Instruction Tuning and In-Context Fine-tuning methods. Our results show that the over-refusal problem is mitigated by In-context fine-tuning. but magnified by R-tuning. However, we also find that the refusal ability may conflict with the quality of the answer. Finally, we develop a simple yet effective refusal method for refusal post-trained models to improve their overall answer quality in terms of refusal and correct answers. Our study provides a more comprehensive understanding of the influence of important factors on RALM systems.

Method: DischargeSim simulates multi-turn conversations between LLM-driven DoctorAgents and PatientAgents with diverse psychosocial profiles across six discharge topics, evaluated on dialogue quality, personalized document generation, and patient comprehension.

Result: Experiments across 18 LLMs reveal significant gaps in discharge education capability, with performance varying widely across patient profiles. Model size doesn’t always yield better education outcomes.

Conclusion: DischargeSim provides a first step toward benchmarking LLMs in post-visit clinical education and promoting equitable, personalized patient support.

Abstract: Discharge communication is a critical yet underexplored component of patient care, where the goal shifts from diagnosis to education. While recent large language model (LLM) benchmarks emphasize in-visit diagnostic reasoning, they fail to evaluate models’ ability to support patients after the visit. We introduce DischargeSim, a novel benchmark that evaluates LLMs on their ability to act as personalized discharge educators. DischargeSim simulates post-visit, multi-turn conversations between LLM-driven DoctorAgents and PatientAgents with diverse psychosocial profiles (e.g., health literacy, education, emotion). Interactions are structured across six clinically grounded discharge topics and assessed along three axes: (1) dialogue quality via automatic and LLM-as-judge evaluation, (2) personalized document generation including free-text summaries and structured AHRQ checklists, and (3) patient comprehension through a downstream multiple-choice exam. Experiments across 18 LLMs reveal significant gaps in discharge education capability, with performance varying widely across patient profiles. Notably, model size does not always yield better education outcomes, highlighting trade-offs in strategy use and content prioritization. DischargeSim offers a first step toward benchmarking LLMs in post-visit clinical education and promoting equitable, personalized patient support.

Method: Applied Biber’s multidimensional analysis (MDA) to human-written texts (BE-21 corpus) and comparable AI-generated texts (AI-Brown corpus), replicated analysis on Czech using AI-Koditex corpus, examined 16 frontier LLMs with various settings and prompts.

Result: Identified systematic differences between human and AI writing across interpretable dimensions, with emphasis on differences between base models and instruction-tuned models.

Conclusion: Created a benchmark that allows models to be compared and ranked based on interpretable dimensions of register variation, providing a systematic way to evaluate LLM performance relative to human writing patterns.

Abstract: This study investigates the register variation in texts written by humans and comparable texts produced by large language models (LLMs). Biber’s multidimensional analysis (MDA) is applied to a sample of human-written texts and AI-created texts generated to be their counterparts to find the dimensions of variation in which LLMs differ most significantly and most systematically from humans. As textual material, a new LLM-generated corpus AI-Brown is used, which is comparable to BE-21 (a Brown family corpus representing contemporary British English). Since all languages except English are underrepresented in the training data of frontier LLMs, similar analysis is replicated on Czech using AI-Koditex corpus and Czech multidimensional model. Examined were 16 frontier models in various settings and prompts, with emphasis placed on the difference between base models and instruction-tuned models. Based on this, a benchmark is created through which models can be compared with each other and ranked in interpretable dimensions.

Method: SENTRA is a supervised Transformer-based encoder that uses selected-next-token-probability sequences and contrastive pre-training on large amounts of unlabeled data.

Result: Experiments on three public datasets across 24 text domains show SENTRA significantly outperforms popular baselines in out-of-domain detection scenarios.

Conclusion: SENTRA demonstrates strong performance as a general-purpose classifier for detecting LLM-generated text, particularly in challenging out-of-domain settings.

Abstract: LLMs are becoming increasingly capable and widespread. Consequently, the potential and reality of their misuse is also growing. In this work, we address the problem of detecting LLM-generated text that is not explicitly declared as such. We present a novel, general-purpose, and supervised LLM text detector, SElected-Next-Token tRAnsformer (SENTRA). SENTRA is a Transformer-based encoder leveraging selected-next-token-probability sequences and utilizing contrastive pre-training on large amounts of unlabeled data. Our experiments on three popular public datasets across 24 domains of text demonstrate SENTRA is a general-purpose classifier that significantly outperforms popular baselines in the out-of-domain setting.

Method: Creates multiple evaluator agents from LLMs, each with specific prompts and topic contexts. Agents independently generate rubrics and conduct evaluations, then engage in roundtable-style dialectical reasoning to reach consensus on final scores.

Result: RES outperforms prior zero-shot AES approaches, achieving up to 34.86% improvement in average QWK over vanilla prompting methods on the ASAP dataset using ChatGPT and Claude.

Conclusion: The multi-agent collaborative framework enables more human-aligned essay scoring by incorporating diverse evaluation perspectives through simulated discussions.

Abstract: The emergence of large language models (LLMs) has brought a new paradigm to automated essay scoring (AES), a long-standing and practical application of natural language processing in education. However, achieving human-level multi-perspective understanding and judgment remains a challenge. In this work, we propose Roundtable Essay Scoring (RES), a multi-agent evaluation framework designed to perform precise and human-aligned scoring under a zero-shot setting. RES constructs evaluator agents based on LLMs, each tailored to a specific prompt and topic context. Each agent independently generates a trait-based rubric and conducts a multi-perspective evaluation. Then, by simulating a roundtable-style discussion, RES consolidates individual evaluations through a dialectical reasoning process to produce a final holistic score that more closely aligns with human evaluation. By enabling collaboration and consensus among agents with diverse evaluation perspectives, RES outperforms prior zero-shot AES approaches. Experiments on the ASAP dataset using ChatGPT and Claude show that RES achieves up to a 34.86% improvement in average QWK over straightforward prompting (Vanilla) methods.

Method: Proposes Empathy-R1 framework with Chain-of-Empathy reasoning (emotions-causes-intentions) inspired by cognitive-behavioral therapy, using two-stage training: Supervised Fine-Tuning followed by RL with a reward model, trained on Empathy-QA dataset.

Result: Achieves strong automatic metrics and human evaluation superiority with 44.30% Win@1 rate, demonstrating interpretable and contextually appropriate responses.

Conclusion: Empathy-R1 represents significant advancement in responsible AI for mental health by enabling transparent, nuanced, and genuinely beneficial support responses.

Abstract: Empathy is critical for effective mental health support, especially when addressing Long Counseling Texts (LCTs). However, existing Large Language Models (LLMs) often generate replies that are semantically fluent but lack the structured reasoning necessary for genuine psychological support, particularly in a Chinese context. To bridge this gap, we introduce Empathy-R1, a novel framework that integrates a Chain-of-Empathy (CoE) reasoning process with Reinforcement Learning (RL) to enhance response quality for LCTs. Inspired by cognitive-behavioral therapy, our CoE paradigm guides the model to sequentially reason about a help-seeker’s emotions, causes, and intentions, making its thinking process both transparent and interpretable. Our framework is empowered by a new large-scale Chinese dataset, Empathy-QA, and a two-stage training process. First, Supervised Fine-Tuning instills the CoE’s reasoning structure. Subsequently, RL, guided by a dedicated reward model, refines the therapeutic relevance and contextual appropriateness of the final responses. Experiments show that Empathy-R1 achieves strong performance on key automatic metrics. More importantly, human evaluations confirm its superiority, showing a clear preference over strong baselines and achieving a Win@1 rate of 44.30% on our new benchmark. By enabling interpretable and contextually nuanced responses, Empathy-R1 represents a significant advancement in developing responsible and genuinely beneficial AI for mental health support.

Method: Introduces LLM2VEC4CXR (domain-adapted LLM encoder for chest X-ray reports) and LLM2CLIP4CXR (dual-tower framework coupling the LLM encoder with vision backbone). Trained on 1.6M CXR studies from public and private sources with heterogeneous reports.

Result: LLM2VEC4CXR improves clinical text understanding over BERT-based baselines, handles abbreviations and style variation, achieves strong clinical alignment. LLM2CLIP4CXR boosts retrieval accuracy and clinically oriented scores with stronger cross-dataset generalization than prior medical CLIP variants.

Conclusion: Robustness - not scale alone - is key to effective multimodal learning in medical imaging. Models demonstrate that LLM encoders provide robust clinical representations that transfer across diverse report styles and better guide image-text alignment.

Abstract: Vision-language pretraining has advanced image-text alignment, yet progress in radiology remains constrained by the heterogeneity of clinical reports, including abbreviations, impression-only notes, and stylistic variability. Unlike general-domain settings where more data often leads to better performance, naively scaling to large collections of noisy reports can plateau or even degrade model learning. We ask whether large language model (LLM) encoders can provide robust clinical representations that transfer across diverse styles and better guide image-text alignment. We introduce LLM2VEC4CXR, a domain-adapted LLM encoder for chest X-ray reports, and LLM2CLIP4CXR, a dual-tower framework that couples this encoder with a vision backbone. LLM2VEC4CXR improves clinical text understanding over BERT-based baselines, handles abbreviations and style variation, and achieves strong clinical alignment on report-level metrics. LLM2CLIP4CXR leverages these embeddings to boost retrieval accuracy and clinically oriented scores, with stronger cross-dataset generalization than prior medical CLIP variants. Trained on 1.6M CXR studies from public and private sources with heterogeneous and noisy reports, our models demonstrate that robustness – not scale alone – is the key to effective multimodal learning. We release models to support further research in medical image-text representation learning.

Method: ORCA operates via an Observe-Reason-Critique-Act loop, querying multiple visual tools with evidential questions, validating cross-model inconsistencies, and refining predictions iteratively without requiring model internals or retraining.

Result: ORCA improves LVLM performance by +3.64% to +40.67% on hallucination benchmarks, achieves +20.11% accuracy gain under adversarial perturbations, and further improves performance when combined with defense techniques.

Conclusion: ORCA offers a promising path toward building more reliable and robust multimodal systems through agentic reasoning without model retraining.

Abstract: Large Vision-Language Models (LVLMs) exhibit strong multimodal capabilities but remain vulnerable to hallucinations from intrinsic errors and adversarial attacks from external exploitations, limiting their reliability in real-world applications. We present ORCA, an agentic reasoning framework that improves the factual accuracy and adversarial robustness of pretrained LVLMs through test-time structured inference reasoning with a suite of small vision models (less than 3B parameters). ORCA operates via an Observe–Reason–Critique–Act loop, querying multiple visual tools with evidential questions, validating cross-model inconsistencies, and refining predictions iteratively without access to model internals or retraining. ORCA also stores intermediate reasoning traces, which supports auditable decision-making. Though designed primarily to mitigate object-level hallucinations, ORCA also exhibits emergent adversarial robustness without requiring adversarial training or defense mechanisms. We evaluate ORCA across three settings: (1) clean images on hallucination benchmarks, (2) adversarially perturbed images without defense, and (3) adversarially perturbed images with defense applied. On the POPE hallucination benchmark, ORCA improves standalone LVLM performance by +3.64% to +40.67% across different subsets. Under adversarial perturbations on POPE, ORCA achieves an average accuracy gain of +20.11% across LVLMs. When combined with defense techniques on adversarially perturbed AMBER images, ORCA further improves standalone LVLM performance, with gains ranging from +1.20% to +48.00% across evaluation metrics. These results demonstrate that ORCA offers a promising path toward building more reliable and robust multimodal systems.

Method: ViSpec employs a lightweight vision adaptor module to compress image tokens into compact representations integrated into the draft model’s attention mechanism. It extracts global feature vectors for input images and augments subsequent text tokens with these features. The framework uses a specialized training dataset created by repurposing existing datasets and generating extended outputs with modified prompts.

Result: ViSpec achieves, to the authors’ knowledge, the first substantial speedup in VLM speculative decoding, overcoming the limitations of previous methods that achieved only modest speedups (<1.5x).

Conclusion: The proposed ViSpec framework successfully addresses the challenge of accelerating vision-language models through speculative decoding by effectively filtering redundant image information layer by layer while maintaining multimodal coherence, representing a significant advancement in VLM acceleration techniques.

Abstract: Speculative decoding is a widely adopted technique for accelerating inference in large language models (LLMs), yet its application to vision-language models (VLMs) remains underexplored, with existing methods achieving only modest speedups (<1.5x). This gap is increasingly significant as multimodal capabilities become central to large-scale models. We hypothesize that large VLMs can effectively filter redundant image information layer by layer without compromising textual comprehension, whereas smaller draft models struggle to do so. To address this, we introduce Vision-Aware Speculative Decoding (ViSpec), a novel framework tailored for VLMs. ViSpec employs a lightweight vision adaptor module to compress image tokens into a compact representation, which is seamlessly integrated into the draft model’s attention mechanism while preserving original image positional information. Additionally, we extract a global feature vector for each input image and augment all subsequent text tokens with this feature to enhance multimodal coherence. To overcome the scarcity of multimodal datasets with long assistant responses, we curate a specialized training dataset by repurposing existing datasets and generating extended outputs using the target VLM with modified prompts. Our training strategy mitigates the risk of the draft model exploiting direct access to the target model’s hidden states, which could otherwise lead to shortcut learning when training solely on target model outputs. Extensive experiments validate ViSpec, achieving, to our knowledge, the first substantial speedup in VLM speculative decoding.

Method: Proposes M-PACE framework using mother-child MLLM setup where a stronger parent model evaluates outputs of smaller child models. Applied to advertisement compliance with 15+ attributes.

Result: Achieved 31x cost reduction (from $0.0159 to $0.0005 per image) with comparable accuracy using Gemini 2.0 Flash as child model selected by mother MLLM.

Conclusion: M-PACE demonstrates effective automation of quality control, significantly reducing human reviewer dependence and enabling real-time compliance assessment at lower costs.

Abstract: Ensuring that multi-modal content adheres to brand, legal, or platform-specific compliance standards is an increasingly complex challenge across domains. Traditional compliance frameworks typically rely on disjointed, multi-stage pipelines that integrate separate modules for image classification, text extraction, audio transcription, hand-crafted checks, and rule-based merges. This architectural fragmentation increases operational overhead, hampers scalability, and hinders the ability to adapt to dynamic guidelines efficiently. With the emergence of Multimodal Large Language Models (MLLMs), there is growing potential to unify these workflows under a single, general-purpose framework capable of jointly processing visual and textual content. In light of this, we propose Multimodal Parameter Agnostic Compliance Engine (M-PACE), a framework designed for assessing attributes across vision-language inputs in a single pass. As a representative use case, we apply M-PACE to advertisement compliance, demonstrating its ability to evaluate over 15 compliance-related attributes. To support structured evaluation, we introduce a human-annotated benchmark enriched with augmented samples that simulate challenging real-world conditions, including visual obstructions and profanity injection. M-PACE employs a mother-child MLLM setup, demonstrating that a stronger parent MLLM evaluating the outputs of smaller child models can significantly reduce dependence on human reviewers, thereby automating quality control. Our analysis reveals that inference costs reduce by over 31 times, with the most efficient models (Gemini 2.0 Flash as child MLLM selected by mother MLLM) operating at 0.0005 per image, compared to 0.0159 for Gemini 2.5 Pro with comparable accuracy, highlighting the trade-off between cost and output quality achieved in real time by M-PACE in real life deployment over advertising data.

Method: Uses Hungarian algorithm and Kalman filtering for data fusion, Past Time Attention (PTA) module for interaction capture, mode attention module for prediction diversity, and anchor-oriented decoder for trajectory generation.

Result: Achieves state-of-the-art performance on DAIR-V2X-Seq dataset, demonstrating effectiveness in cooperative trajectory prediction for V2X scenarios.

Conclusion: CoPAD effectively addresses limitations of single-vehicle perception through cooperative multi-source data fusion and attention mechanisms, providing high-quality trajectory predictions for autonomous driving applications.

Abstract: Recently, data-driven trajectory prediction methods have achieved remarkable results, significantly advancing the development of autonomous driving. However, the instability of single-vehicle perception introduces certain limitations to trajectory prediction. In this paper, a novel lightweight framework for cooperative trajectory prediction, CoPAD, is proposed. This framework incorporates a fusion module based on the Hungarian algorithm and Kalman filtering, along with the Past Time Attention (PTA) module, mode attention module and anchor-oriented decoder (AoD). It effectively performs early fusion on multi-source trajectory data from vehicles and road infrastructure, enabling the trajectories with high completeness and accuracy. The PTA module can efficiently capture potential interaction information among historical trajectories, and the mode attention module is proposed to enrich the diversity of predictions. Additionally, the decoder based on sparse anchors is designed to generate the final complete trajectories. Extensive experiments show that CoPAD achieves the state-of-the-art performance on the DAIR-V2X-Seq dataset, validating the effectiveness of the model in cooperative trajectory prediction in V2X scenarios.

Method: Developed a virtual AFM framework to generate synthetic multi-view AFM images (~542,000 proteins), trained a conditional diffusion model for novel view synthesis, and used instance-specific Neural Radiance Field (NeRF) for 3D reconstruction.

Result: Reconstructed 3D protein structures achieve average Chamfer Distance within AFM imaging resolution, demonstrating high structural fidelity. Validated on experimental AFM images of various protein complexes.

Conclusion: ProFusion shows strong potential for accurate, cost-effective protein complex structure prediction and enables rapid iterative validation using AFM experiments.

Abstract: AI-based in silico methods have improved protein structure prediction but often struggle with large protein complexes (PCs) involving multiple interacting proteins due to missing 3D spatial cues. Experimental techniques like Cryo-EM are accurate but costly and time-consuming. We present ProFusion, a hybrid framework that integrates a deep learning model with Atomic Force Microscopy (AFM), which provides high-resolution height maps from random orientations, naturally yielding multi-view data for 3D reconstruction. However, generating a large-scale AFM imaging data set sufficient to train deep learning models is impractical. Therefore, we developed a virtual AFM framework that simulates the imaging process and generated a dataset of ~542,000 proteins with multi-view synthetic AFM images. We train a conditional diffusion model to synthesize novel views from unposed inputs and an instance-specific Neural Radiance Field (NeRF) model to reconstruct 3D structures. Our reconstructed 3D protein structures achieve an average Chamfer Distance within the AFM imaging resolution, reflecting high structural fidelity. Our method is extensively validated on experimental AFM images of various PCs, demonstrating strong potential for accurate, cost-effective protein complex structure prediction and rapid iterative validation using AFM experiments.

Method: Builds on Grad-eclip with novel Hierarchical Semantic Relationship Module featuring multi-scale feature processing, adaptive attention weighting, and cross-modal alignment to capture relationships at different granularities.

Result: Produces more focused and contextually aware visualizations that better reflect how models process complex scenes, generalizing across various domains.

Conclusion: MMEL offers valuable insights into model decisions for applications requiring high interpretability and reliability through improved gradient-based attribution maps.

Abstract: Recent advances in vision-language models have significantly expanded the frontiers of automated image analysis. However, applying these models in safety-critical contexts remains challenging due to the complex relationships between objects, subtle visual cues, and the heightened demand for transparency and reliability. This paper presents the Multi-Modal Explainable Learning (MMEL) framework, designed to enhance the interpretability of vision-language models while maintaining high performance. Building upon prior work in gradient-based explanations for transformer architectures (Grad-eclip), MMEL introduces a novel Hierarchical Semantic Relationship Module that enhances model interpretability through multi-scale feature processing, adaptive attention weighting, and cross-modal alignment. Our approach processes features at multiple semantic levels to capture relationships between image regions at different granularities, applying learnable layer-specific weights to balance contributions across the model’s depth. This results in more comprehensive visual explanations that highlight both primary objects and their contextual relationships with improved precision. Through extensive experiments on standard datasets, we demonstrate that by incorporating semantic relationship information into gradient-based attribution maps, MMEL produces more focused and contextually aware visualizations that better reflect how vision-language models process complex scenes. The MMEL framework generalizes across various domains, offering valuable insights into model decisions for applications requiring high interpretability and reliability.

Method: Leverages individual 3D shapes to create multi-object scenes with explicit spatial relations, pairs them with coherent descriptions refined by LLM, and augments contrastive training with structured compositional samples.

Result: Substantial performance gains in zero-shot classification, few-shot part segmentation, 3D visual question answering, and spatial reasoning across multiple datasets and encoder architectures.

Conclusion: SceneForge’s compositional augmentations are model-agnostic and effectively enhance 3D-text alignment, demonstrating robust generalization to complex scenarios and spatial reasoning capabilities.

Abstract: The whole is greater than the sum of its parts-even in 3D-text contrastive learning. We introduce SceneForge, a novel framework that enhances contrastive alignment between 3D point clouds and text through structured multi-object scene compositions. SceneForge leverages individual 3D shapes to construct multi-object scenes with explicit spatial relations, pairing them with coherent multi-object descriptions refined by a large language model. By augmenting contrastive training with these structured, compositional samples, SceneForge effectively addresses the scarcity of large-scale 3D-text datasets, significantly enriching data complexity and diversity. We systematically investigate critical design elements, such as the optimal number of objects per scene, the proportion of compositional samples in training batches, and scene construction strategies. Extensive experiments demonstrate that SceneForge delivers substantial performance gains across multiple tasks, including zero-shot classification on ModelNet, ScanObjNN, Objaverse-LVIS, and ScanNet, as well as few-shot part segmentation on ShapeNetPart. SceneForge’s compositional augmentations are model-agnostic, consistently improving performance across multiple encoder architectures. Moreover, SceneForge improves 3D visual question answering on ScanQA, generalizes robustly to retrieval scenarios with increasing scene complexity, and showcases spatial reasoning capabilities by adapting spatial configurations to align precisely with textual instructions.

Method: NAP pre-filters tokens into foreground/background using navigation traits (e.g., filtering image views based on navigable directions) and uses LLMs to extract navigation-relevant instructions. It focuses pruning on background tokens and removes low-importance navigation nodes to prevent backtracking.

Result: Experiments on standard VLN benchmarks show NAP significantly outperforms prior work, preserving higher success rates while saving more than 50% FLOPS.

Conclusion: Navigation-aware pruning effectively addresses VLN-specific efficiency challenges by leveraging domain knowledge to minimize performance loss while achieving substantial computational savings.

Abstract: Large models achieve strong performance on Vision-and-Language Navigation (VLN) tasks, but are costly to run in resource-limited environments. Token pruning offers appealing tradeoffs for efficiency with minimal performance loss by reducing model input size, but prior work overlooks VLN-specific challenges. For example, information loss from pruning can effectively increase computational cost due to longer walks. Thus, the inability to identify uninformative tokens undermines the supposed efficiency gains from pruning. To address this, we propose Navigation-Aware Pruning (NAP), which uses navigation-specific traits to simplify the pruning process by pre-filtering tokens into foreground and background. For example, image views are filtered based on whether the agent can navigate in that direction. We also extract navigation-relevant instructions using a Large Language Model. After filtering, we focus pruning on background tokens, minimizing information loss. To further help avoid increases in navigation length, we discourage backtracking by removing low-importance navigation nodes. Experiments on standard VLN benchmarks show NAP significantly outperforms prior work, preserving higher success rates while saving more than 50% FLOPS.

Method: Proposes Grounding via View Retrieval (GVR) framework that leverages object-level view retrieval to collect grounding clues from multiple views, transforming 3DVG into a 2D retrieval task.

Result: Extensive experiments show GVR achieves state-of-the-art visual grounding performance while avoiding per-scene training, providing a foundation for zero-shot 3DVG research.

Conclusion: GVR offers an effective zero-shot solution for 3D visual grounding that eliminates costly 3D annotations and per-scene training requirements.

Abstract: 3D Visual Grounding (3DVG) aims to locate objects in 3D scenes based on text prompts, which is essential for applications such as robotics. However, existing 3DVG methods encounter two main challenges: first, they struggle to handle the implicit representation of spatial textures in 3D Gaussian Splatting (3DGS), making per-scene training indispensable; second, they typically require larges amounts of labeled data for effective training. To this end, we propose \underline{G}rounding via \underline{V}iew \underline{R}etrieval (GVR), a novel zero-shot visual grounding framework for 3DGS to transform 3DVG as a 2D retrieval task that leverages object-level view retrieval to collect grounding clues from multiple views, which not only avoids the costly process of 3D annotation, but also eliminates the need for per-scene training. Extensive experiments demonstrate that our method achieves state-of-the-art visual grounding performance while avoiding per-scene training, providing a solid foundation for zero-shot 3DVG research. Video demos can be found in https://github.com/leviome/GVR_demos.

Method: Proposes RespoDiff with two distinct learnable modules: one for capturing/enforcing responsible concepts (fairness/safety) and another for maintaining semantic alignment with neutral prompts. Uses a novel score-matching objective for effective coordination between modules.

Result: Outperforms state-of-the-art methods, improving responsible and semantically coherent generation by 20% across diverse, unseen prompts. Successfully integrates with large-scale models like SDXL while enhancing fairness and safety.

Conclusion: RespoDiff provides an effective framework for responsible text-to-image generation that maintains high image quality and semantic fidelity while significantly improving fairness and safety metrics.

Abstract: The rapid advancement of diffusion models has enabled high-fidelity and semantically rich text-to-image generation; however, ensuring fairness and safety remains an open challenge. Existing methods typically improve fairness and safety at the expense of semantic fidelity and image quality. In this work, we propose RespoDiff, a novel framework for responsible text-to-image generation that incorporates a dual-module transformation on the intermediate bottleneck representations of diffusion models. Our approach introduces two distinct learnable modules: one focused on capturing and enforcing responsible concepts, such as fairness and safety, and the other dedicated to maintaining semantic alignment with neutral prompts. To facilitate the dual learning process, we introduce a novel score-matching objective that enables effective coordination between the modules. Our method outperforms state-of-the-art methods in responsible generation by ensuring semantic alignment while optimizing both objectives without compromising image fidelity. Our approach improves responsible and semantically coherent generation by 20% across diverse, unseen prompts. Moreover, it integrates seamlessly into large-scale models like SDXL, enhancing fairness and safety. Code will be released upon acceptance.

Method: Uses flow matching generative model with learned orthogonal motion latent space instead of pixel-based latent space. Introduces transformer-based vector field predictor with frame-wise conditioning mechanism and supports speech-driven emotion enhancement.

Result: Extensive experiments show FLOAT outperforms state-of-the-art audio-driven talking portrait methods in visual quality, motion fidelity, and efficiency.

Conclusion: The method enables efficient generation and editing of temporally consistent motion for talking portrait videos with natural expressive motion incorporation.

Abstract: With the rapid advancement of diffusion-based generative models, portrait image animation has achieved remarkable results. However, it still faces challenges in temporally consistent video generation and fast sampling due to its iterative sampling nature. This paper presents FLOAT, an audio-driven talking portrait video generation method based on flow matching generative model. Instead of a pixel-based latent space, we take advantage of a learned orthogonal motion latent space, enabling efficient generation and editing of temporally consistent motion. To achieve this, we introduce a transformer-based vector field predictor with an effective frame-wise conditioning mechanism. Additionally, our method supports speech-driven emotion enhancement, enabling a natural incorporation of expressive motions. Extensive experiments demonstrate that our method outperforms state-of-the-art audio-driven talking portrait methods in terms of visual quality, motion fidelity, and efficiency.

Method: Integrated JEST and autoguidance into unified codebase for benchmarking, evaluated on 2-D synthetic data and 3x64x64 image generation tasks, comparing methods at equal wall-clock time and equal sample counts while accounting for selection overhead.

Result: Autoguidance consistently improved sample quality and diversity. Early AJEST (selection only at training start) matched or modestly exceeded autoguidance in data efficiency, but its time overhead made autoguidance or uniform random selection preferable in most cases.

Conclusion: Targeted online selection can yield early training efficiency gains, but robust quality improvements are primarily driven by autoguidance. Data selection may be beneficial in specific scenarios despite its complexity and overhead.

Abstract: The costs of generative model compute rekindled promises and hopes for efficient data curation. In this work, we investigate whether recently developed autoguidance and online data selection methods can improve the time and sample efficiency of training generative diffusion models. We integrate joint example selection (JEST) and autoguidance into a unified code base for fast ablation and benchmarking. We evaluate combinations of data curation on a controlled 2-D synthetic data generation task as well as (3x64x64)-D image generation. Our comparisons are made at equal wall-clock time and equal number of samples, explicitly accounting for the overhead of selection. Across experiments, autoguidance consistently improves sample quality and diversity. Early AJEST (applying selection only at the beginning of training) can match or modestly exceed autoguidance alone in data efficiency on both tasks. However, its time overhead and added complexity make autoguidance or uniform random data selection preferable in most situations. These findings suggest that while targeted online selection can yield efficiency gains in early training, robust sample quality improvements are primarily driven by autoguidance. We discuss limitations and scope, and outline when data selection may be beneficial.

Method: Revisiting tactile representation learning framework by incorporating material-aware priors - pre-learned characteristics specific to different materials that help capture surface texture nuances.

Result: The method enables more accurate and contextually rich tactile feedback across diverse materials and textures.

Conclusion: Material-aware priors improve tactile representation learning performance for real-world applications like robotics, haptic feedback systems, and material editing.

Abstract: Tactile perception is profoundly influenced by the surface properties of objects in contact. However, despite their crucial role in shaping tactile experiences, these material characteristics have been largely neglected in existing tactile representation learning methods. Most approaches primarily focus on aligning tactile data with visual or textual information, overlooking the richness of tactile feedback that comes from understanding the materials’ inherent properties. In this work, we address this gap by revisiting the tactile representation learning framework and incorporating material-aware priors into the learning process. These priors, which represent pre-learned characteristics specific to different materials, allow tactile models to better capture and generalize the nuances of surface texture. Our method enables more accurate, contextually rich tactile feedback across diverse materials and textures, improving performance in real-world applications such as robotics, haptic feedback systems, and material editing.

Method: PRISM uses radial reduction of discrete Fourier transform to capture model-specific signatures from amplitude and phase information, followed by linear discriminant analysis clustering for attribution.

Result: PRISM achieves 92.04% attribution accuracy on PRISM-36K dataset, 81.60% average accuracy on four benchmarks, and 95.06% accuracy on GenImage for real vs fake detection.

Conclusion: Frequency-domain fingerprinting is effective for cross-architecture and cross-dataset model attribution, providing a viable solution for accountability and trust in generative AI systems.

Abstract: A critical need has emerged for generative AI: attribution methods. That is, solutions that can identify the model originating AI-generated content. This feature, generally relevant in multimodal applications, is especially sensitive in commercial settings where users subscribe to paid proprietary services and expect guarantees about the source of the content they receive. To address these issues, we introduce PRISM, a scalable Phase-enhanced Radial-based Image Signature Mapping framework for fingerprinting AI-generated images. PRISM is based on a radial reduction of the discrete Fourier transform that leverages amplitude and phase information to capture model-specific signatures. The output of the above process is subsequently clustered via linear discriminant analysis to achieve reliable model attribution in diverse settings, even if the model’s internal details are inaccessible. To support our work, we construct PRISM-36K, a novel dataset of 36,000 images generated by six text-to-image GAN- and diffusion-based models. On this dataset, PRISM achieves an attribution accuracy of 92.04%. We additionally evaluate our method on four benchmarks from the literature, reaching an average accuracy of 81.60%. Finally, we evaluate our methodology also in the binary task of detecting real vs fake images, achieving an average accuracy of 88.41%. We obtain our best result on GenImage with an accuracy of 95.06%, whereas the original benchmark achieved 82.20%. Our results demonstrate the effectiveness of frequency-domain fingerprinting for cross-architecture and cross-dataset model attribution, offering a viable solution for enforcing accountability and trust in generative AI systems.

Method: Evaluated multiple vision transformer models across various mental rotation tasks including block structures, text, and photo-realistic objects, probing representations layer by layer.

Result: Self-supervised ViTs outperform supervised ViTs in capturing geometric structure; intermediate layers show better performance than final layers; task difficulty patterns mirror human reaction times.

Conclusion: Vision transformers develop mental rotation abilities with similar constraints to humans, suggesting neural networks can capture fundamental spatial reasoning capabilities.

Abstract: Mental rotation is a key test of spatial reasoning in humans and has been central to understanding how perception supports cognition. Despite the success of modern vision transformers, it is still unclear how well these models develop similar abilities. In this work, we present a systematic evaluation of ViT, CLIP, DINOv2, and DINOv3 across a range of mental-rotation tasks, from simple block structures similar to those used by Shepard and Metzler to study human cognition, to more complex block figures, three types of text, and photo-realistic objects. By probing model representations layer by layer, we examine where and how these networks succeed. We find that i) self-supervised ViTs capture geometric structure better than supervised ViTs; ii) intermediate layers perform better than final layers; iii) task difficulty increases with rotation complexity and occlusion, mirroring human reaction times and suggesting similar constraints in embedding space representations.

Method: Uses a pure transformer architecture with 4D rotary position embeddings, adversarial-free training with perceptual and Gram matrix losses, and progressive training curriculum from single images to videos and 3D.

Result: Achieves 0.21 rFID with 82.2% ImageNet accuracy for images, 3.01 rFVD with 40.2% MSRVTT retrieval for videos, and 28.28 PSNR with 90.9% classification accuracy for 3D.

Conclusion: AToken enables competitive performance in both generation and understanding tasks, paving the way for next-generation multimodal AI systems with unified visual tokenization.

Abstract: We present AToken, the first unified visual tokenizer that achieves both high-fidelity reconstruction and semantic understanding across images, videos, and 3D assets. Unlike existing tokenizers that specialize in either reconstruction or understanding for single modalities, AToken encodes these diverse visual inputs into a shared 4D latent space, unifying both tasks and modalities in a single framework. Specifically, we introduce a pure transformer architecture with 4D rotary position embeddings to process visual inputs of arbitrary resolutions and temporal durations. To ensure stable training, we introduce an adversarial-free training objective that combines perceptual and Gram matrix losses, achieving state-of-the-art reconstruction quality. By employing a progressive training curriculum, AToken gradually expands from single images, videos, and 3D, and supports both continuous and discrete latent tokens. AToken achieves 0.21 rFID with 82.2% ImageNet accuracy for images, 3.01 rFVD with 40.2% MSRVTT retrieval for videos, and 28.28 PSNR with 90.9% classification accuracy for 3D.. In downstream applications, AToken enables both visual generation tasks (e.g., image generation with continuous and discrete tokens, text-to-video generation, image-to-3D synthesis) and understanding tasks (e.g., multimodal LLMs), achieving competitive performance across all benchmarks. These results shed light on the next-generation multimodal AI systems built upon unified visual tokenization.

Method: The authors evaluate unmodified ViT features (keys, queries, values, and post-feed-forward features) using simple classification rules like hyperplane-based (logistic regression) and cosine-similarity based approaches, without additional feature transformations.

Result: The analysis provides insights into optimal token type and decision rule choices based on task, context, and pre-training objective, with detailed findings on two widely-used datasets.

Conclusion: The study demonstrates that unaltered ViT features have strong intrinsic representation capabilities for both standard and few-shot learning scenarios in image classification and segmentation tasks.

Abstract: Self-Supervised Learning (SSL) for Vision Transformers (ViTs) has recently demonstrated considerable potential as a pre-training strategy for a variety of computer vision tasks, including image classification and segmentation, both in standard and few-shot downstream contexts. Two pre-training objectives dominate the landscape of SSL techniques: Contrastive Learning and Masked Image Modeling. Features (or tokens) extracted from the final transformer attention block – specifically, the keys, queries, and values – as well as features obtained after the final block’s feed-forward layer, have become a common foundation for addressing downstream tasks. However, in many existing approaches, these pre-trained ViT features are further processed through additional transformation layers, often involving lightweight heads or combined with distillation, to achieve superior task performance. Although such methods can improve task outcomes, to the best of our knowledge, a comprehensive analysis of the intrinsic representation capabilities of unaltered ViT features has yet to be conducted. This study aims to bridge this gap by systematically evaluating the use of these unmodified features across image classification and segmentation tasks, in both standard and few-shot contexts. The classification and segmentation rules that we use are either hyperplane based (as in logistic regression) or cosine-similarity based, both of which rely on the presence of interpretable directions in the ViT’s latent space. Based on the previous rules and without the use of additional feature transformations, we conduct an analysis across token types, tasks, and pre-trained ViT models. This study provides insights into the optimal choice for token type and decision rule based on the task, context, and the pre-training objective, while reporting detailed findings on two widely-used datasets.

Method: Developed FoMER benchmark with 1.1k samples across 10 tasks and 8 embodiments covering 3 robot types, featuring a novel evaluation framework that separates perceptual grounding from action reasoning.

Result: Empirical analysis reveals both potential and current limitations of LMMs in embodied reasoning, identifying key challenges for future robot intelligence research.

Conclusion: The benchmark provides a foundation for advancing embodied AI research, with data and code made publicly available to facilitate further development in this domain.

Abstract: Embodied agents operating in the physical world must make decisions that are not only effective but also safe, spatially coherent, and grounded in context. While recent advances in large multimodal models (LMMs) have shown promising capabilities in visual understanding and language generation, their ability to perform structured reasoning for real-world embodied tasks remains underexplored. In this work, we aim to understand how well foundation models can perform step-by-step reasoning in embodied environments. To this end, we propose the Foundation Model Embodied Reasoning (FoMER) benchmark, designed to evaluate the reasoning capabilities of LMMs in complex embodied decision-making scenarios. Our benchmark spans a diverse set of tasks that require agents to interpret multimodal observations, reason about physical constraints and safety, and generate valid next actions in natural language. We present (i) a large-scale, curated suite of embodied reasoning tasks, (ii) a novel evaluation framework that disentangles perceptual grounding from action reasoning, and (iii) empirical analysis of several leading LMMs under this setting. Our benchmark includes over 1.1k samples with detailed step-by-step reasoning across 10 tasks and 8 embodiments, covering three different robot types. Our results highlight both the potential and current limitations of LMMs in embodied reasoning, pointing towards key challenges and opportunities for future research in robot intelligence. Our data and code will be made publicly available.

Method: AdaptiveNN uses a theory integrating representation learning with self-rewarding reinforcement learning, enabling end-to-end training without additional supervision on fixation locations. It progressively identifies and attends to task-relevant regions through sequential fixations.

Result: On 17 benchmarks spanning 9 tasks, AdaptiveNN achieves up to 28x inference cost reduction without sacrificing accuracy, adapts to varying task demands and resource budgets without retraining, and provides enhanced interpretability via fixation patterns.

Conclusion: AdaptiveNN demonstrates a promising avenue toward efficient, flexible, and interpretable computer vision, exhibiting human-like perceptual behaviors and potential as a tool for investigating visual cognition.

Abstract: Human vision is highly adaptive, efficiently sampling intricate environments by sequentially fixating on task-relevant regions. In contrast, prevailing machine vision models passively process entire scenes at once, resulting in excessive resource demands scaling with spatial-temporal input resolution and model size, yielding critical limitations impeding both future advancements and real-world application. Here we introduce AdaptiveNN, a general framework aiming to drive a paradigm shift from ‘passive’ to ‘active, adaptive’ vision models. AdaptiveNN formulates visual perception as a coarse-to-fine sequential decision-making process, progressively identifying and attending to regions pertinent to the task, incrementally combining information across fixations, and actively concluding observation when sufficient. We establish a theory integrating representation learning with self-rewarding reinforcement learning, enabling end-to-end training of the non-differentiable AdaptiveNN without additional supervision on fixation locations. We assess AdaptiveNN on 17 benchmarks spanning 9 tasks, including large-scale visual recognition, fine-grained discrimination, visual search, processing images from real driving and medical scenarios, language-driven embodied AI, and side-by-side comparisons with humans. AdaptiveNN achieves up to 28x inference cost reduction without sacrificing accuracy, flexibly adapts to varying task demands and resource budgets without retraining, and provides enhanced interpretability via its fixation patterns, demonstrating a promising avenue toward efficient, flexible, and interpretable computer vision. Furthermore, AdaptiveNN exhibits closely human-like perceptual behaviors in many cases, revealing its potential as a valuable tool for investigating visual cognition. Code is available at https://github.com/LeapLabTHU/AdaptiveNN.

Method: Proposes CoDoL with Domain Meta Network (DMN) to generate input-conditional tokens using domain information, capturing both instance-specific and domain-specific information for better prompts.

Result: Extensive experiments on four OOD benchmarks (PACS, VLCS, OfficeHome, DigitDG) show CoDoL effectively improves vision-language embedding alignment and OOD generalization performance.

Conclusion: CoDoL successfully addresses CLIP’s limitations by leveraging domain information for better prompt learning and embedding alignment, demonstrating strong OOD generalization capabilities.

Abstract: Recent advances in pre-training vision-language models (VLMs), e.g., contrastive language-image pre-training (CLIP) methods, have shown great potential in learning out-of-distribution (OOD) representations. Despite showing competitive performance, the prompt-based CLIP methods still suffer from: i) inaccurate text descriptions, which leads to degraded accuracy and robustness, and poses a challenge for zero-shot CLIP methods. ii) limited vision-language embedding alignment, which significantly affects the generalization performance. To tackle the above issues, this paper proposes a novel Conditional Domain prompt Learning (CoDoL) method, which utilizes readily-available domain information to form prompts and improves the vision-language embedding alignment for improving OOD generalization. To capture both instance-specific and domain-specific information, we further propose a lightweight Domain Meta Network (DMN) to generate input-conditional tokens for images in each domain. Extensive experiments on four OOD benchmarks (PACS, VLCS, OfficeHome and DigitDG) validate the effectiveness of our proposed CoDoL in terms of improving the vision-language embedding alignment as well as the out-of-distribution generalization performance.

Method: Proposes LowDiff with a cascaded approach generating increasingly higher resolution outputs using a unified model that progressively refines images from low to desired resolution.

Result: Achieved over 50% throughput improvement across all datasets while maintaining comparable or better quality. Specific results include FID of 2.11 and IS of 9.87 on unconditional CIFAR-10, FID of 1.94 and IS of 10.03 on conditional CIFAR-10, FID of 2.43 on FFHQ 64x64, and FID of 4.00 with IS of 195.06 on ImageNet 256x256.

Conclusion: LowDiff demonstrates effectiveness and generality across various datasets and settings, providing substantial efficiency gains without compromising image quality, applicable to both pixel and latent space diffusion models.

Abstract: Diffusion models have achieved remarkable success in image generation but their practical application is often hindered by the slow sampling speed. Prior efforts of improving efficiency primarily focus on compressing models or reducing the total number of denoising steps, largely neglecting the possibility to leverage multiple input resolutions in the generation process. In this work, we propose LowDiff, a novel and efficient diffusion framework based on a cascaded approach by generating increasingly higher resolution outputs. Besides, LowDiff employs a unified model to progressively refine images from low resolution to the desired resolution. With the proposed architecture design and generation techniques, we achieve comparable or even superior performance with much fewer high-resolution sampling steps. LowDiff is applicable to diffusion models in both pixel space and latent space. Extensive experiments on both conditional and unconditional generation tasks across CIFAR-10, FFHQ and ImageNet demonstrate the effectiveness and generality of our method. Results show over 50% throughput improvement across all datasets and settings while maintaining comparable or better quality. On unconditional CIFAR-10, LowDiff achieves an FID of 2.11 and IS of 9.87, while on conditional CIFAR-10, an FID of 1.94 and IS of 10.03. On FFHQ 64x64, LowDiff achieves an FID of 2.43, and on ImageNet 256x256, LowDiff built on LightningDiT-B/1 produces high-quality samples with a FID of 4.00 and an IS of 195.06, together with substantial efficiency gains.

Method: Proposes MaskAttn-SDXL which learns binary masks per layer and injects them into cross-attention logits before softmax to sparsify token-to-latent interactions, preserving only semantically relevant connections without requiring positional encodings, auxiliary tokens, or external region masks.

Result: The model improves spatial compliance and attribute binding in multi-object prompts while preserving overall image quality and diversity with negligible inference overhead.

Conclusion: Logit-level masked cross-attention is a data-efficient primitive for enforcing compositional control, making MaskAttn-SDXL a practical extension for spatial control in text-to-image generation.

Abstract: Text-to-image diffusion models achieve impressive realism but often suffer from compositional failures on prompts with multiple objects, attributes, and spatial relations, resulting in cross-token interference where entities entangle, attributes mix across objects, and spatial cues are violated. To address these failures, we propose MaskAttn-SDXL,a region-level gating mechanism applied to the cross-attention logits of Stable Diffusion XL(SDXL)’s UNet. MaskAttn-SDXL learns a binary mask per layer, injecting it into each cross-attention logit map before softmax to sparsify token-to-latent interactions so that only semantically relevant connections remain active. The method requires no positional encodings, auxiliary tokens, or external region masks, and preserves the original inference path with negligible overhead. In practice, our model improves spatial compliance and attribute binding in multi-object prompts while preserving overall image quality and diversity. These findings demonstrate that logit-level maksed cross-attention is an data-efficient primitve for enforcing compositional control, and our method thus serves as a practical extension for spatial control in text-to-image generation.

Method: RaceGAN uses a novel framework that maps style codes across multiple domains during racial attribute translation, enabling style mapping without reference images while preserving individuality and high-level semantics.

Result: RaceGAN outperforms other models in translating racial features (Asian, White, Black) on the Chicago Face Dataset, with quantitative validation using InceptionReNetv2-based classification. The model also effectively partitions latent space into distinct ethnic group clusters.

Conclusion: RaceGAN provides an effective solution for racial attribute translation that maintains individual identity while achieving superior performance in multi-domain racial feature mapping without requiring reference images.

Abstract: Generative adversarial networks (GANs) have demonstrated significant progress in unpaired image-to-image translation in recent years for several applications. CycleGAN was the first to lead the way, although it was restricted to a pair of domains. StarGAN overcame this constraint by tackling image-to-image translation across various domains, although it was not able to map in-depth low-level style changes for these domains. Style mapping via reference-guided image synthesis has been made possible by the innovations of StarGANv2 and StyleGAN. However, these models do not maintain individuality and need an extra reference image in addition to the input. Our study aims to translate racial traits by means of multi-domain image-to-image translation. We present RaceGAN, a novel framework capable of mapping style codes over several domains during racial attribute translation while maintaining individuality and high level semantics without relying on a reference image. RaceGAN outperforms other models in translating racial features (i.e., Asian, White, and Black) when tested on Chicago Face Dataset. We also give quantitative findings utilizing InceptionReNetv2-based classification to demonstrate the effectiveness of our racial translation. Moreover, we investigate how well the model partitions the latent space into distinct clusters of faces for each ethnic group.

Method: Leverages user-defined part labels from a limited set of images and efficiently transfers them to a larger dataset, aggregating part-based local explanations to generate global symbolic explanations.

Result: Enables the generation of human-understandable explanations for model decisions on a large scale without incurring significant labeling costs.

Conclusion: The approach provides an efficient way to create global symbolic explanations for deep learning models, making them more interpretable while minimizing annotation efforts.

Abstract: Deep learning models are notoriously opaque. Existing explanation methods often focus on localized visual explanations for individual images. Concept-based explanations, while offering global insights, require extensive annotations, incurring significant labeling cost. We propose an approach that leverages user-defined part labels from a limited set of images and efficiently transfers them to a larger dataset. This enables the generation of global symbolic explanations by aggregating part-based local explanations, ultimately providing human-understandable explanations for model decisions on a large scale.

Method: A causality-decoupling framework that disentangles causal fingerprints from image-specific content and style in a semantic-invariant latent space derived from pre-trained diffusion reconstruction residuals. The approach enhances fingerprint granularity with diverse feature representations.

Result: Experiments show the approach outperforms existing methods in model attribution across representative GANs and diffusion models. The method also achieves source anonymization using counterfactual examples generated from causal fingerprints.

Conclusion: The proposed causal fingerprint framework demonstrates strong potential for forgery detection, model copyright tracing, and identity protection, validating the importance of causality in model attribution.

Abstract: AI generative models leave implicit traces in their generated images, which are commonly referred to as model fingerprints and are exploited for source attribution. Prior methods rely on model-specific cues or synthesis artifacts, yielding limited fingerprints that may generalize poorly across different generative models. We argue that a complete model fingerprint should reflect the causality between image provenance and model traces, a direction largely unexplored. To this end, we conceptualize the \emph{causal fingerprint} of generative models, and propose a causality-decoupling framework that disentangles it from image-specific content and style in a semantic-invariant latent space derived from pre-trained diffusion reconstruction residual. We further enhance fingerprint granularity with diverse feature representations. We validate causality by assessing attribution performance across representative GANs and diffusion models and by achieving source anonymization using counterfactual examples generated from causal fingerprints. Experiments show our approach outperforms existing methods in model attribution, indicating strong potential for forgery detection, model copyright tracing, and identity protection.

Method: Proposes iCBIR-Sli which uses a series of 2D slices and effectively aggregates slice information to achieve low-dimensional representations with high completeness, usability, robustness, and interoperability.

Result: Achieved top-1 retrieval performance (macro F1 = 0.859) on five brain MR datasets (ADNI2/3, OASIS3/4, AIBL) for Alzheimer’s disease and cognitively normal cases, comparable to classification-specific deep learning models without external classifiers.

Conclusion: iCBIR-Sli provides an effective and interpretable CBIR solution that identifies disease-indicative brain regions while achieving competitive performance without requiring large training datasets or external classifiers.

Abstract: Current methods for searching brain MR images rely on text-based approaches, highlighting a significant need for content-based image retrieval (CBIR) systems. Directly applying 3D brain MR images to machine learning models offers the benefit of effectively learning the brain’s structure; however, building the generalized model necessitates a large amount of training data. While models that consider depth direction and utilize continuous 2D slices have demonstrated success in segmentation and classification tasks involving 3D data, concerns remain. Specifically, using general 2D slices may lead to the oversight of pathological features and discontinuities in depth direction information. Furthermore, to the best of the authors’ knowledge, there have been no attempts to develop a practical CBIR system that preserves the entire brain’s structural information. In this study, we propose an interpretable CBIR method for brain MR images, named iCBIR-Sli (Interpretable CBIR with 2D Slice Embedding), which, for the first time globally, utilizes a series of 2D slices. iCBIR-Sli addresses the challenges associated with using 2D slices by effectively aggregating slice information, thereby achieving low-dimensional representations with high completeness, usability, robustness, and interoperability, which are qualities essential for effective CBIR. In retrieval evaluation experiments utilizing five publicly available brain MR datasets (ADNI2/3, OASIS3/4, AIBL) for Alzheimer’s disease and cognitively normal, iCBIR-Sli demonstrated top-1 retrieval performance (macro F1 = 0.859), comparable to existing deep learning models explicitly designed for classification, without the need for an external classifier. Additionally, the method provided high interpretability by clearly identifying the brain regions indicative of the searched-for disease.

Method: Pretrained self-supervised backbones (BYOL, DINO, MAE, MoCo) on multi-institutional brain tumor MRI and applied distributionally robust optimization (DRO) to mitigate site and class imbalance. Evaluated on molecular classification of common/uncommon markers and survival prediction.

Result: Improved molecular prediction and reduced site-specific embedding differences. Mean balanced accuracy increased from 0.744 to 0.785 and AUC from 0.656 to 0.676 at CUIMC, with largest gains for underrepresented endpoints. Survival c-index improved at all sites (CUIMC: 0.592 to 0.597, UPenn: 0.647 to 0.672, UCSF: 0.600 to 0.627).

Conclusion: Coupling foundation models with DRO yields more site-invariant representations, improves prediction of common and uncommon markers, and enhances survival discrimination, highlighting the need for prospective validation and integration of longitudinal/interventional signals for precision neuro-oncology.

Abstract: Neuro-oncology poses unique challenges for machine learning due to heterogeneous data and tumor complexity, limiting the ability of foundation models (FMs) to generalize across cohorts. Existing FMs also perform poorly in predicting uncommon molecular markers, which are essential for treatment response and risk stratification. To address these gaps, we developed a neuro-oncology specific FM with a distributionally robust loss function, enabling accurate estimation of tumor phenotypes while maintaining cross-institution generalization. We pretrained self-supervised backbones (BYOL, DINO, MAE, MoCo) on multi-institutional brain tumor MRI and applied distributionally robust optimization (DRO) to mitigate site and class imbalance. Downstream tasks included molecular classification of common markers (MGMT, IDH1, 1p/19q, EGFR), uncommon alterations (ATRX, TP53, CDKN2A/2B, TERT), continuous markers (Ki-67, TP53), and overall survival prediction in IDH1 wild-type glioblastoma at UCSF, UPenn, and CUIMC. Our method improved molecular prediction and reduced site-specific embedding differences. At CUIMC, mean balanced accuracy rose from 0.744 to 0.785 and AUC from 0.656 to 0.676, with the largest gains for underrepresented endpoints (CDKN2A/2B accuracy 0.86 to 0.92, AUC 0.73 to 0.92; ATRX AUC 0.69 to 0.82; Ki-67 accuracy 0.60 to 0.69). For survival, c-index improved at all sites: CUIMC 0.592 to 0.597, UPenn 0.647 to 0.672, UCSF 0.600 to 0.627. Grad-CAM highlighted tumor and peri-tumoral regions, confirming interpretability. Overall, coupling FMs with DRO yields more site-invariant representations, improves prediction of common and uncommon markers, and enhances survival discrimination, underscoring the need for prospective validation and integration of longitudinal and interventional signals to advance precision neuro-oncology.

Method: Proposes DSDNet with Synergic Attention with Dynamic Modulation (SADM) module for raw-to-YCbCr mapping and Luminance-Chrominance Adaptive Transformer (LCAT) to decouple luminance and chrominance representations for better color fidelity.

Result: DSDNet outperforms state-of-the-art methods in both visual quality and quantitative evaluation, achieving 2.4x faster inference speed than the second-best method.

Conclusion: The proposed single-stage raw domain framework effectively addresses moiré removal while maintaining luminance and color fidelity, offering practical advantages for mobile imaging applications.

Abstract: With the rapid advancement of mobile imaging, capturing screens using smartphones has become a prevalent practice in distance learning and conference recording. However, moir'e artifacts, caused by frequency aliasing between display screens and camera sensors, are further amplified by the image signal processing pipeline, leading to severe visual degradation. Existing sRGB domain demoir'eing methods struggle with irreversible information loss, while recent two-stage raw domain approaches suffer from information bottlenecks and inference inefficiency. To address these limitations, we propose a single-stage raw domain demoir'eing framework, Dual-Stream Demoir'eing Network (DSDNet), which leverages the synergy of raw and YCbCr images to remove moir'e while preserving luminance and color fidelity. Specifically, to guide luminance correction and moir'e removal, we design a raw-to-YCbCr mapping pipeline and introduce the Synergic Attention with Dynamic Modulation (SADM) module. This module enriches the raw-to-sRGB conversion with cross-domain contextual features. Furthermore, to better guide color fidelity, we develop a Luminance-Chrominance Adaptive Transformer (LCAT), which decouples luminance and chrominance representations. Extensive experiments demonstrate that DSDNet outperforms state-of-the-art methods in both visual quality and quantitative evaluation and achieves an inference speed $\mathrm{\textbf{2.4x}}$ faster than the second-best method, highlighting its practical advantages. We provide an anonymous online demo at https://xxxxxxxxdsdnet.github.io/DSDNet/.

Method: RAD-Conv employs four boundary offsets per kernel element to create dynamic rectangular regions that adjust size and shape to match image content, decoupling receptive field shape from kernel structure.

Result: The approach enables capture of both local details and long-range dependencies even with small 1x1 kernels, combining attention-like adaptability with convolution efficiency.

Conclusion: RAD-Conv provides a practical solution for building more expressive and efficient vision models, bridging the gap between rigid convolutional architectures and computationally expensive attention-based methods.

Abstract: We introduce Region-Aware Deformable Convolution (RAD-Conv), a new convolutional operator that enhances neural networks’ ability to adapt to complex image structures. Unlike traditional deformable convolutions, which are limited to fixed quadrilateral sampling areas, RAD-Conv uses four boundary offsets per kernel element to create flexible, rectangular regions that dynamically adjust their size and shape to match image content. This approach allows precise control over the receptive field’s width and height, enabling the capture of both local details and long-range dependencies, even with small 1x1 kernels. By decoupling the receptive field’s shape from the kernel’s structure, RAD-Conv combines the adaptability of attention mechanisms with the efficiency of standard convolutions. This innovative design offers a practical solution for building more expressive and efficient vision models, bridging the gap between rigid convolutional architectures and computationally costly attention-based methods.

Method: Single transformer architecture trained to learn shared representations between grayscale images, DEMs, surface normals, and albedo maps, supporting flexible translation between any input and target modality.

Result: The foundation model learns physically plausible relations across the four modalities and successfully formulates image-based 3D reconstruction and albedo estimation as multimodal learning problems.

Conclusion: Multimodal learning shows strong potential for solving Shape and Albedo from Shading problems and provides a new approach for large-scale planetary 3D reconstruction, with future improvements expected from adding more input modalities.

Abstract: Multimodal learning is an emerging research topic across multiple disciplines but has rarely been applied to planetary science. In this contribution, we propose a single, unified transformer architecture trained to learn shared representations between multiple sources like grayscale images, Digital Elevation Models (DEMs), surface normals, and albedo maps. The architecture supports flexible translation from any input modality to any target modality. Our results demonstrate that our foundation model learns physically plausible relations across these four modalities. We further identify that image-based 3D reconstruction and albedo estimation (Shape and Albedo from Shading) of lunar images can be formulated as a multimodal learning problem. Our results demonstrate the potential of multimodal learning to solve Shape and Albedo from Shading and provide a new approach for large-scale planetary 3D reconstruction. Adding more input modalities in the future will further improve the results and enable tasks such as photometric normalization and co-registration.

Method: Proposes a native edge-centric formulation modeling wall segments as directed, geometrically continuous edges. Uses a dual-query transformer decoder with perturbed and latent queries within a denoising framework for stable optimization and faster convergence.

Result: Achieves state-of-the-art performance with F1 scores of 99.1% (rooms), 91.7% (corners), and 89.3% (angles) on Structured3D and SceneCAD datasets. Demonstrates strong cross-dataset generalization.

Conclusion: CAGE provides an effective architectural innovation for robust floorplan reconstruction, ensuring watertight, topologically valid room boundaries while reducing artifacts and improving geometric continuity.

Abstract: We present \textbf{CAGE} (\textit{Continuity-Aware edGE}) network, a \textcolor{red}{robust} framework for reconstructing vector floorplans directly from point-cloud density maps. Traditional corner-based polygon representations are highly sensitive to noise and incomplete observations, often resulting in fragmented or implausible layouts. Recent line grouping methods leverage structural cues to improve robustness but still struggle to recover fine geometric details. To address these limitations, we propose a \textit{native} edge-centric formulation, modeling each wall segment as a directed, geometrically continuous edge. This representation enables inference of coherent floorplan structures, ensuring watertight, topologically valid room boundaries while improving robustness and reducing artifacts. Towards this design, we develop a dual-query transformer decoder that integrates perturbed and latent queries within a denoising framework, which not only stabilizes optimization but also accelerates convergence. Extensive experiments on Structured3D and SceneCAD show that \textbf{CAGE} achieves state-of-the-art performance, with F1 scores of 99.1% (rooms), 91.7% (corners), and 89.3% (angles). The method also demonstrates strong cross-dataset generalization, underscoring the efficacy of our architectural innovations. Code and pretrained models will be released upon acceptance.

Method: Use self-supervised learning with joint embedding predictive architecture (JEPA) pretraining on unlabeled CT scans and electronic health records, followed by supervised finetuning.

Result: Outperforms unregularized multimodal and imaging-only models internally (0.91 vs 0.88 vs 0.73 AUC) but underperforms externally (0.72 vs 0.75 AUC). Developed synthetic environment to analyze JEPA limitations.

Conclusion: Innovative approach leveraging unlabeled multimodal archives improves predictive models for pulmonary nodule diagnosis, but has limitations in external validation that require further investigation.

Abstract: The development of multimodal models for pulmonary nodule diagnosis is limited by the scarcity of labeled data and the tendency for these models to overfit on the training distribution. In this work, we leverage self-supervised learning from longitudinal and multimodal archives to address these challenges. We curate an unlabeled set of patients with CT scans and linked electronic health records from our home institution to power joint embedding predictive architecture (JEPA) pretraining. After supervised finetuning, we show that our approach outperforms an unregularized multimodal model and imaging-only model in an internal cohort (ours: 0.91, multimodal: 0.88, imaging-only: 0.73 AUC), but underperforms in an external cohort (ours: 0.72, imaging-only: 0.75 AUC). We develop a synthetic environment that characterizes the context in which JEPA may underperform. This work innovates an approach that leverages unlabeled multimodal medical archives to improve predictive models and demonstrates its advantages and limitations in pulmonary nodule diagnosis.

Method: Proposes a novel generative model training workflow with bi-directional contrastive loss and diversity loss, plus a caption synthesis strategy to improve text-to-image retrieval performance.

Result: Superior performance and efficiency on Flickr30K, COCO, and CC3M datasets, achieving results 18x faster than state-of-the-art methods.

Conclusion: EDGE provides an efficient solution for multimodal dataset distillation by addressing key challenges in generative model training for image-text datasets.

Abstract: Dataset distillation aims to synthesize a small dataset from a large dataset, enabling the model trained on it to perform well on the original dataset. With the blooming of large language models and multimodal large language models, the importance of multimodal datasets, particularly image-text datasets, has grown significantly. However, existing multimodal dataset distillation methods are constrained by the Matching Training Trajectories algorithm, which significantly increases the computing resource requirement, and takes days to process the distillation. In this work, we introduce EDGE, a generative distillation method for efficient multimodal dataset distillation. Specifically, we identify two key challenges of distilling multimodal datasets with generative models: 1) The lack of correlation between generated images and captions. 2) The lack of diversity among generated samples. To address the aforementioned issues, we propose a novel generative model training workflow with a bi-directional contrastive loss and a diversity loss. Furthermore, we propose a caption synthesis strategy to further improve text-to-image retrieval performance by introducing more text information. Our method is evaluated on Flickr30K, COCO, and CC3M datasets, demonstrating superior performance and efficiency compared to existing approaches. Notably, our method achieves results 18x faster than the state-of-the-art method.

Method: Builds a modular system with three components (image tokenizer, world model, video decoder) using pre-trained open-source models fine-tuned on publicly available automotive data (BDD100K) with streamlined interfaces.

Result: Achieves realistic driving scene video prediction at 256x256 resolution and 4 fps with only one frame of algorithmic latency.

Conclusion: OpenViGA provides an open, reproducible, and academically scalable alternative to proprietary video generation systems, with publicly available code and models.

Abstract: Recent successful video generation systems that predict and create realistic automotive driving scenes from short video inputs assign tokenization, future state prediction (world model), and video decoding to dedicated models. These approaches often utilize large models that require significant training resources, offer limited insight into design choices, and lack publicly available code and datasets. In this work, we address these deficiencies and present OpenViGA, an open video generation system for automotive driving scenes. Our contributions are: Unlike several earlier works for video generation, such as GAIA-1, we provide a deep analysis of the three components of our system by separate quantitative and qualitative evaluation: Image tokenizer, world model, video decoder. Second, we purely build upon powerful pre-trained open source models from various domains, which we fine-tune by publicly available automotive data (BDD100K) on GPU hardware at academic scale. Third, we build a coherent video generation system by streamlining interfaces of our components. Fourth, due to public availability of the underlying models and data, we allow full reproducibility. Finally, we also publish our code and models on Github. For an image size of 256x256 at 4 fps we are able to predict realistic driving scene videos frame-by-frame with only one frame of algorithmic latency.

Method: Used representational similarity analysis on H&E image patches from TCGA to compare six CPath foundation models spanning vision-language contrastive learning (CONCH, PLIP, KEEP) and self-distillation (UNI v2, Virchow v2, Prov-GigaPath) approaches.

Result: UNI2 and Virchow2 had the most distinct representational structures; Prov-GigaPath had highest average similarity across models; same training paradigm didn’t guarantee higher similarity; all models showed high slide-dependence but low disease-dependence; stain normalization reduced slide-dependence by 5.5-20.5%; vision-language models had more compact representations than vision-only models.

Conclusion: The findings highlight opportunities to improve robustness to slide-specific features, inform model ensembling strategies, and provide insights into how training paradigms shape model representations, with an extendable framework for medical imaging domains.

Abstract: Foundation models are increasingly developed in computational pathology (CPath) given their promise in facilitating many downstream tasks. While recent studies have evaluated task performance across models, less is known about the structure and variability of their learned representations. Here, we systematically analyze the representational spaces of six CPath foundation models using techniques popularized in computational neuroscience. The models analyzed span vision-language contrastive learning (CONCH, PLIP, KEEP) and self-distillation (UNI (v2), Virchow (v2), Prov-GigaPath) approaches. Through representational similarity analysis using H&E image patches from TCGA, we find that UNI2 and Virchow2 have the most distinct representational structures, whereas Prov-Gigapath has the highest average similarity across models. Having the same training paradigm (vision-only vs. vision-language) did not guarantee higher representational similarity. The representations of all models showed a high slide-dependence, but relatively low disease-dependence. Stain normalization decreased slide-dependence for all models by a range of 5.5% (CONCH) to 20.5% (PLIP). In terms of intrinsic dimensionality, vision-language models demonstrated relatively compact representations, compared to the more distributed representations of vision-only models. These findings highlight opportunities to improve robustness to slide-specific features, inform model ensembling strategies, and provide insights into how training paradigms shape model representations. Our framework is extendable across medical imaging domains, where probing the internal representations of foundation models can help ensure effective development and deployment.

Method: A compact vision-language architecture that explicitly incorporates region-level spatial reasoning using RGB and depth cues, employing a three-stage curriculum to progressively align visual and language features, enable spatial relationship understanding, and adapt to task-specific datasets.

Result: With only 600M parameters, SmolRGPT achieves competitive results on challenging warehouse spatial reasoning benchmarks, matching or exceeding the performance of much larger alternatives.

Conclusion: The findings highlight the potential for efficient, deployable multimodal intelligence in real-world settings without sacrificing core spatial reasoning capabilities.

Abstract: Recent advances in vision-language models (VLMs) have enabled powerful multimodal reasoning, but state-of-the-art approaches typically rely on extremely large models with prohibitive computational and memory requirements. This makes their deployment challenging in resource-constrained environments such as warehouses, robotics, and industrial applications, where both efficiency and robust spatial understanding are critical. In this work, we present SmolRGPT, a compact vision-language architecture that explicitly incorporates region-level spatial reasoning by integrating both RGB and depth cues. SmolRGPT employs a three-stage curriculum that progressively align visual and language features, enables spatial relationship understanding, and adapts to task-specific datasets. We demonstrate that with only 600M parameters, SmolRGPT achieves competitive results on challenging warehouse spatial reasoning benchmarks, matching or exceeding the performance of much larger alternatives. These findings highlight the potential for efficient, deployable multimodal intelligence in real-world settings without sacrificing core spatial reasoning capabilities. The code of the experimentation will be available at: https://github.com/abtraore/SmolRGPT

Method: Built on Diffusion Transformer (DiT) foundation model with two adapters: ID-adapter using Perceiver Resampler for facial embeddings, and Ref-adapter integrating dense VAE features through cross-attention across transformer layers.

Result: Superior face resemblance, competitive prompt following, and strong video quality demonstrated on 800 test cases with 40 subjects and 20 prompts.

Conclusion: Lynx advances the state of personalized video generation by effectively preserving identity while maintaining temporal coherence and visual quality.

Abstract: We present Lynx, a high-fidelity model for personalized video synthesis from a single input image. Built on an open-source Diffusion Transformer (DiT) foundation model, Lynx introduces two lightweight adapters to ensure identity fidelity. The ID-adapter employs a Perceiver Resampler to convert ArcFace-derived facial embeddings into compact identity tokens for conditioning, while the Ref-adapter integrates dense VAE features from a frozen reference pathway, injecting fine-grained details across all transformer layers through cross-attention. These modules collectively enable robust identity preservation while maintaining temporal coherence and visual realism. Through evaluation on a curated benchmark of 40 subjects and 20 unbiased prompts, which yielded 800 test cases, Lynx has demonstrated superior face resemblance, competitive prompt following, and strong video quality, thereby advancing the state of personalized video generation.

Method: Bi-level optimization that jointly learns selection variables, sparse invertible mask, and sample-specific perturbations. Inner problem synthesizes triggers using inverse mask, outer problem refines mask to suppress backdoor behavior.

Result: Outperforms existing pruning-based approaches, maintains strong performance with limited data, achieves competitive results compared to state-of-the-art fine-tuning methods.

Conclusion: The approach is particularly effective in restoring correct predictions for compromised samples after backdoor mitigation, offering interpretability and robustness advantages.

Abstract: Model pruning has gained traction as a promising defense strategy against backdoor attacks in deep learning. However, existing pruning-based approaches often fall short in accurately identifying and removing the specific parameters responsible for inducing backdoor behaviors. Despite the dominance of fine-tuning-based defenses in recent literature, largely due to their superior performance, pruning remains a compelling alternative, offering greater interpretability and improved robustness in low-data regimes. In this paper, we propose a novel pruning approach featuring a learned \emph{selection} mechanism to identify parameters critical to both main and backdoor tasks, along with an \emph{invertible} pruning mask designed to simultaneously achieve two complementary goals: eliminating the backdoor task while preserving it through the inverse mask. We formulate this as a bi-level optimization problem that jointly learns selection variables, a sparse invertible mask, and sample-specific backdoor perturbations derived from clean data. The inner problem synthesizes candidate triggers using the inverse mask, while the outer problem refines the mask to suppress backdoor behavior without impairing clean-task accuracy. Extensive experiments demonstrate that our approach outperforms existing pruning-based backdoor mitigation approaches, maintains strong performance under limited data conditions, and achieves competitive results compared to state-of-the-art fine-tuning approaches. Notably, the proposed approach is particularly effective in restoring correct predictions for compromised samples after successful backdoor mitigation.

Method: Leverages minimal coding length in lossy data coding as a surrogate for entropy, with a scalable reformulation based on Mixture of Experts (MOE) for fast computation and handling long-tailed distributions of weights/activations.

Result: Pushes QAT limits to x-bit activation for the first time, with accuracy comparable to or surpassing full precision counterparts. Establishes new state-of-the-art for QAT across various computer vision tasks.

Conclusion: MEC-Quant provides a principled approach to quantization that explicitly optimizes representation structure, successfully addressing bias issues in low-bit quantization and achieving superior performance.

Abstract: Quantization-Aware Training (QAT) has driven much attention to produce efficient neural networks. Current QAT still obtains inferior performances compared with the Full Precision (FP) counterpart. In this work, we argue that quantization inevitably introduce biases into the learned representation, especially under the extremely low-bit setting. To cope with this issue, we propose Maximum Entropy Coding Quantization (MEC-Quant), a more principled objective that explicitly optimizes on the structure of the representation, so that the learned representation is less biased and thus generalizes better to unseen in-distribution samples. To make the objective end-to-end trainable, we propose to leverage the minimal coding length in lossy data coding as a computationally tractable surrogate for the entropy, and further derive a scalable reformulation of the objective based on Mixture Of Experts (MOE) that not only allows fast computation but also handles the long-tailed distribution for weights or activation values. Extensive experiments on various tasks on computer vision tasks prove its superiority. With MEC-Qaunt, the limit of QAT is pushed to the x-bit activation for the first time and the accuracy of MEC-Quant is comparable to or even surpass the FP counterpart. Without bells and whistles, MEC-Qaunt establishes a new state of the art for QAT.

Method: Proposes GUI-ARP framework with Adaptive Region Perception (ARP) and Adaptive Stage Controlling (ASC) that dynamically crops task-relevant regions and adapts inference strategy. Uses two-phase training with supervised fine-tuning and reinforcement fine-tuning based on Group Relative Policy Optimization (GRPO).

Result: Achieves state-of-the-art performance: 60.8% accuracy on ScreenSpot-Pro and 30.9% on UI-Vision benchmark with 7B model. GUI-ARP-7B competes strongly against larger 72B models (UI-TARS-72B at 38.1%) and proprietary models.

Conclusion: GUI-ARP demonstrates effective adaptive multi-stage inference for GUI grounding, achieving competitive performance with smaller models through intelligent region perception and stage control mechanisms.

Abstract: Existing GUI grounding methods often struggle with fine-grained localization in high-resolution screenshots. To address this, we propose GUI-ARP, a novel framework that enables adaptive multi-stage inference. Equipped with the proposed Adaptive Region Perception (ARP) and Adaptive Stage Controlling (ASC), GUI-ARP dynamically exploits visual attention for cropping task-relevant regions and adapts its inference strategy, performing a single-stage inference for simple cases and a multi-stage analysis for more complex scenarios. This is achieved through a two-phase training pipeline that integrates supervised fine-tuning with reinforcement fine-tuning based on Group Relative Policy Optimization (GRPO). Extensive experiments demonstrate that the proposed GUI-ARP achieves state-of-the-art performance on challenging GUI grounding benchmarks, with a 7B model reaching 60.8% accuracy on ScreenSpot-Pro and 30.9% on UI-Vision benchmark. Notably, GUI-ARP-7B demonstrates strong competitiveness against open-source 72B models (UI-TARS-72B at 38.1%) and proprietary models.

Method: Extract intermediate features from diffusion-Transformer models at specific timesteps and blocks, use a heuristic local-search algorithm to find optimal image-text block-timestep pairs, and fuse features via linear projection and addition.

Result: Achieved 98.6% mAP on MS-COCO-enhanced and 45.7% mAP on Visual Genome 500, surpassing CNN, graph, and Transformer baselines. Found that Layer 12 consistently performs best for images, and t-SNE shows tighter semantic clusters.

Conclusion: Diff-Feat demonstrates that intermediate features from diffusion-Transformers are highly effective for multi-label classification, with consistent patterns across datasets and superior performance over existing methods.

Abstract: Multi-label classification has broad applications and depends on powerful representations capable of capturing multi-label interactions. We introduce \textit{Diff-Feat}, a simple but powerful framework that extracts intermediate features from pre-trained diffusion-Transformer models for images and text, and fuses them for downstream tasks. We observe that for vision tasks, the most discriminative intermediate feature along the diffusion process occurs at the middle step and is located in the middle block in Transformer. In contrast, for language tasks, the best feature occurs at the noise-free step and is located in the deepest block. In particular, we observe a striking phenomenon across varying datasets: a mysterious “Layer $12$” consistently yields the best performance on various downstream classification tasks for images (under DiT-XL/2-256$\times$256). We devise a heuristic local-search algorithm that pinpoints the locally optimal “image-text”$\times$“block-timestep” pair among a few candidates, avoiding an exhaustive grid search. A simple fusion-linear projection followed by addition-of the selected representations yields state-of-the-art performance: 98.6% mAP on MS-COCO-enhanced and 45.7% mAP on Visual Genome 500, surpassing strong CNN, graph, and Transformer baselines by a wide margin. t-SNE and clustering metrics further reveal that \textit{Diff-Feat} forms tighter semantic clusters than unimodal counterparts. The code is available at https://github.com/lt-0123/Diff-Feat.

Method: SAMPO integrates temporal causal decoding with bidirectional spatial attention, uses an asymmetric multi-scale tokenizer, and includes a trajectory-aware motion prompt module to inject spatiotemporal cues about object and robot trajectories.

Result: SAMPO achieves competitive performance in action-conditioned video prediction and model-based control, improving generation quality with 4.4× faster inference, and demonstrates strong zero-shot generalization and scaling behavior.

Conclusion: The proposed SAMPO framework effectively addresses limitations of existing world models by preserving spatial locality, supporting parallel decoding, and enhancing dynamic scene understanding through motion prompts and asymmetric tokenization.

Abstract: World models allow agents to simulate the consequences of actions in imagined environments for planning, control, and long-horizon decision-making. However, existing autoregressive world models struggle with visually coherent predictions due to disrupted spatial structure, inefficient decoding, and inadequate motion modeling. In response, we propose \textbf{S}cale-wise \textbf{A}utoregression with \textbf{M}otion \textbf{P}r\textbf{O}mpt (\textbf{SAMPO}), a hybrid framework that combines visual autoregressive modeling for intra-frame generation with causal modeling for next-frame generation. Specifically, SAMPO integrates temporal causal decoding with bidirectional spatial attention, which preserves spatial locality and supports parallel decoding within each scale. This design significantly enhances both temporal consistency and rollout efficiency. To further improve dynamic scene understanding, we devise an asymmetric multi-scale tokenizer that preserves spatial details in observed frames and extracts compact dynamic representations for future frames, optimizing both memory usage and model performance. Additionally, we introduce a trajectory-aware motion prompt module that injects spatiotemporal cues about object and robot trajectories, focusing attention on dynamic regions and improving temporal consistency and physical realism. Extensive experiments show that SAMPO achieves competitive performance in action-conditioned video prediction and model-based control, improving generation quality with 4.4$\times$ faster inference. We also evaluate SAMPO’s zero-shot generalization and scaling behavior, demonstrating its ability to generalize to unseen tasks and benefit from larger model sizes.

Method: Proposes a symmetrical bidirectional framework with text-guided image decoder and image-guided text decoder. Uses low-resolution images for global visual representations and high-resolution images partitioned into sub-images with masked image modeling for fine-grained local features. Adopts mixed-scale image strategy to balance perception and computation.

Result: Outperforms state-of-the-art methods with F1-score improvements: 1.1% in desire understanding, 0.6% in emotion recognition, and 0.9% in sentiment analysis on MSED dataset.

Conclusion: The proposed symmetrical bidirectional multimodal learning framework effectively captures intention-related representations and demonstrates consistent improvements across desire understanding, emotion recognition, and sentiment analysis tasks.

Abstract: Desire, as an intention that drives human behavior, is closely related to both emotion and sentiment. Multimodal learning has advanced sentiment and emotion recognition, but multimodal approaches specially targeting human desire understanding remain underexplored. And existing methods in sentiment analysis predominantly emphasize verbal cues and overlook images as complementary non-verbal cues. To address these gaps, we propose a Symmetrical Bidirectional Multimodal Learning Framework for Desire, Emotion, and Sentiment Recognition, which enforces mutual guidance between text and image modalities to effectively capture intention-related representations in the image. Specifically, low-resolution images are used to obtain global visual representations for cross-modal alignment, while high resolution images are partitioned into sub-images and modeled with masked image modeling to enhance the ability to capture fine-grained local features. A text-guided image decoder and an image-guided text decoder are introduced to facilitate deep cross-modal interaction at both local and global representations of image information. Additionally, to balance perceptual gains with computation cost, a mixed-scale image strategy is adopted, where high-resolution images are cropped into sub-images for masked modeling. The proposed approach is evaluated on MSED, a multimodal dataset that includes a desire understanding benchmark, as well as emotion and sentiment recognition. Experimental results indicate consistent improvements over other state-of-the-art methods, validating the effectiveness of our proposed method. Specifically, our method outperforms existing approaches, achieving F1-score improvements of 1.1% in desire understanding, 0.6% in emotion recognition, and 0.9% in sentiment analysis. Our code is available at: https://github.com/especiallyW/SyDES.

Method: The BTL framework decomposes interactions into three biologically plausible phases: Blink (rapid detection/attention), Think (reasoning/decision-making), and Link (executable command generation). It includes automated blink data generation and a novel rule-based reward mechanism for reinforcement learning.

Result: The developed BTL-UI agent model demonstrates state-of-the-art performance across both static GUI understanding and dynamic interaction tasks in comprehensive benchmarks.

Conclusion: The framework provides conclusive empirical validation for developing advanced GUI agents that better align with natural human cognitive processes.

Abstract: In the field of AI-driven human-GUI interaction automation, while rapid advances in multimodal large language models and reinforcement fine-tuning techniques have yielded remarkable progress, a fundamental challenge persists: their interaction logic significantly deviates from natural human-GUI communication patterns. To fill this gap, we propose “Blink-Think-Link” (BTL), a brain-inspired framework for human-GUI interaction that mimics the human cognitive process between users and graphical interfaces. The system decomposes interactions into three biologically plausible phases: (1) Blink - rapid detection and attention to relevant screen areas, analogous to saccadic eye movements; (2) Think - higher-level reasoning and decision-making, mirroring cognitive planning; and (3) Link - generation of executable commands for precise motor control, emulating human action selection mechanisms. Additionally, we introduce two key technical innovations for the BTL framework: (1) Blink Data Generation - an automated annotation pipeline specifically optimized for blink data, and (2) BTL Reward – the first rule-based reward mechanism that enables reinforcement learning driven by both process and outcome. Building upon this framework, we develop a GUI agent model named BTL-UI, which demonstrates consistent state-of-the-art performance across both static GUI understanding and dynamic interaction tasks in comprehensive benchmarks. These results provide conclusive empirical validation of the framework’s efficacy in developing advanced GUI Agents.

Method: Introduces two components: (1) Video-Language Checker to verify subject/action presence in videos, reducing false positives; (2) Key-Frame Sampler that adaptively selects informative frames for better temporal context.

Result: Achieved J&F score of 64.14% on MeViS test set, ranking 2nd place in RVOS track of 7th LSVOS Challenge at ICCV 2025.

Conclusion: The training-free framework successfully improves Sa2VA’s RVOS performance through explicit verification and adaptive frame sampling, demonstrating effective enhancement without model retraining.

Abstract: Referential Video Object Segmentation (RVOS) aims to segment all objects in a video that match a given natural language description, bridging the gap between vision and language understanding. Recent work, such as Sa2VA, combines Large Language Models (LLMs) with SAM~2, leveraging the strong video reasoning capability of LLMs to guide video segmentation. In this work, we present a training-free framework that substantially improves Sa2VA’s performance on the RVOS task. Our method introduces two key components: (1) a Video-Language Checker that explicitly verifies whether the subject and action described in the query actually appear in the video, thereby reducing false positives; and (2) a Key-Frame Sampler that adaptively selects informative frames to better capture both early object appearances and long-range temporal context. Without any additional training, our approach achieves a J&F score of 64.14% on the MeViS test set, ranking 2nd place in the RVOS track of the 7th LSVOS Challenge at ICCV 2025.

Method: Proposes a Size-Invariant Evaluation (SIEva) framework that evaluates separable components individually and aggregates results, plus a model-agnostic optimization framework (SIOpt) that adheres to size-invariant principles.

Result: The approach effectively mitigates size imbalance impact, enhances detection of salient objects across various sizes, and provides theoretical evidence supporting the validity of the new evaluation protocols.

Conclusion: The proposed size-invariant evaluation and optimization frameworks address fundamental limitations in SOD assessment, offering more balanced performance evaluation and improved detection capabilities for objects of varying sizes.

Abstract: This paper investigates a fundamental yet underexplored issue in Salient Object Detection (SOD): the size-invariant property for evaluation protocols, particularly in scenarios when multiple salient objects of significantly different sizes appear within a single image. We first present a novel perspective to expose the inherent size sensitivity of existing widely used SOD metrics. Through careful theoretical derivations, we show that the evaluation outcome of an image under current SOD metrics can be essentially decomposed into a sum of several separable terms, with the contribution of each term being directly proportional to its corresponding region size. Consequently, the prediction errors would be dominated by the larger regions, while smaller yet potentially more semantically important objects are often overlooked, leading to biased performance assessments and practical degradation. To address this challenge, a generic Size-Invariant Evaluation (SIEva) framework is proposed. The core idea is to evaluate each separable component individually and then aggregate the results, thereby effectively mitigating the impact of size imbalance across objects. Building upon this, we further develop a dedicated optimization framework (SIOpt), which adheres to the size-invariant principle and significantly enhances the detection of salient objects across a broad range of sizes. Notably, SIOpt is model-agnostic and can be seamlessly integrated with a wide range of SOD backbones. Theoretically, we also present generalization analysis of SOD methods and provide evidence supporting the validity of our new evaluation protocols. Finally, comprehensive experiments speak to the efficacy of our proposed approach. The code is available at https://github.com/Ferry-Li/SI-SOD.

Method: Built on 3DGS with geometric priors from monocular depth; uses SfM points to anchor local semantic regions for alignment, and applies geometry-guided supervision at virtual views (fine-grained and coarse) to enforce 3D consistency and mitigate overfitting.

Result: MS-GS achieves photorealistic renderings in challenging sparse-view, multi-appearance conditions, significantly outperforming existing methods across datasets, with a new dataset and benchmark for realistic evaluation.

Conclusion: The framework effectively addresses sparse-view and multi-appearance challenges in scene reconstruction, leveraging geometric and semantic cues to enhance 3DGS performance, validated by superior results and new benchmarks.

Abstract: In-the-wild photo collections often contain limited volumes of imagery and exhibit multiple appearances, e.g., taken at different times of day or seasons, posing significant challenges to scene reconstruction and novel view synthesis. Although recent adaptations of Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) have improved in these areas, they tend to oversmooth and are prone to overfitting. In this paper, we present MS-GS, a novel framework designed with Multi-appearance capabilities in Sparse-view scenarios using 3DGS. To address the lack of support due to sparse initializations, our approach is built on the geometric priors elicited from monocular depth estimations. The key lies in extracting and utilizing local semantic regions with a Structure-from-Motion (SfM) points anchored algorithm for reliable alignment and geometry cues. Then, to introduce multi-view constraints, we propose a series of geometry-guided supervision at virtual views in a fine-grained and coarse scheme to encourage 3D consistency and reduce overfitting. We also introduce a dataset and an in-the-wild experiment setting to set up more realistic benchmarks. We demonstrate that MS-GS achieves photorealistic renderings under various challenging sparse-view and multi-appearance conditions and outperforms existing approaches significantly across different datasets.

Method: HFN introduces a Decision Network that dynamically adjusts modality weights during inference and a Weighted Multi-Modal Feature Fusion module to handle incomplete data. The framework integrates video, audio, and text data for comprehensive analysis.

Result: Experiments on FakeTT and newly collected VESV datasets show improvements of 2.71% and 4.14% in Macro F1 scores over state-of-the-art methods, demonstrating superior performance in short video fake news detection.

Conclusion: This work establishes a robust solution for identifying fake news in short video platforms and contributes the VESV dataset, paving the way for more reliable approaches in combating misinformation.

Abstract: The rapid proliferation of short video platforms has necessitated advanced methods for detecting fake news. This need arises from the widespread influence and ease of sharing misinformation, which can lead to significant societal harm. Current methods often struggle with the dynamic and multimodal nature of short video content. This paper presents HFN, Heterogeneous Fusion Net, a novel multimodal framework that integrates video, audio, and text data to evaluate the authenticity of short video content. HFN introduces a Decision Network that dynamically adjusts modality weights during inference and a Weighted Multi-Modal Feature Fusion module to ensure robust performance even with incomplete data. Additionally, we contribute a comprehensive dataset VESV (VEracity on Short Videos) specifically designed for short video fake news detection. Experiments conducted on the FakeTT and newly collected VESV datasets demonstrate improvements of 2.71% and 4.14% in Marco F1 over state-of-the-art methods. This work establishes a robust solution capable of effectively identifying fake news in the complex landscape of short video platforms, paving the way for more reliable and comprehensive approaches in combating misinformation.

Method: Iterative, interdisciplinary collaboration through early prototyping, shadow deployment, and continuous feedback. Using public datasets and AI models despite domain shift issues, and implementing automated AI-based image quality checks.

Result: Found that treating AI development and workflow digitization as an end-to-end iterative co-design process is crucial. Public datasets and models are useful despite performance limitations due to domain shift.

Conclusion: Documenting practical challenges and lessons learned addresses the gap in contextual, actionable field knowledge for building real-world AI-assisted telehealth and mass-screening programs in resource-constrained settings.

Abstract: Vision- and hearing-threatening diseases cause preventable disability, especially in resource-constrained settings(RCS) with few specialists and limited screening setup. Large scale AI-assisted screening and telehealth has potential to expand early detection, but practical deployment is challenging in paper-based workflows and limited documented field experience exist to build upon. We provide insights on challenges and ways forward in development to adoption of scalable AI-assisted Telehealth and screening in such settings. Specifically, we find that iterative, interdisciplinary collaboration through early prototyping, shadow deployment and continuous feedback is important to build shared understanding as well as reduce usability hurdles when transitioning from paper-based to AI-ready workflows. We find public datasets and AI models highly useful despite poor performance due to domain shift. In addition, we find the need for automated AI-based image quality check to capture gradable images for robust screening in high-volume camps. Our field learning stress the importance of treating AI development and workflow digitization as an end-to-end, iterative co-design process. By documenting these practical challenges and lessons learned, we aim to address the gap in contextual, actionable field knowledge for building real-world AI-assisted telehealth and mass-screening programs in RCS.

Method: The framework integrates two plug-and-play modules: (1) Bi-Temporal Deformable Alignment (BTDA) for geometric correction at semantic feature level, and (2) Scale-Sparse Change Amplifier (SSCA) for selective amplification of high-confidence change signals while suppressing noise.

Result: Experiments show significant improvement over the ChangeMamba baseline, increasing F1-score from 0.5730 to 0.5903 and IoU from 0.4015 to 0.4187.

Conclusion: The ‘align-then-enhance’ strategy effectively addresses both geometric and feature-level challenges in remote sensing change detection, offering a robust and easily deployable solution.

Abstract: Remote sensing change detection (RSCD) is vital for identifying land-cover changes, yet existing methods, including state-of-the-art State Space Models (SSMs), often lack explicit mechanisms to handle geometric misalignments and struggle to distinguish subtle, true changes from noise.To address this, we introduce DC-Mamba, an “align-then-enhance” framework built upon the ChangeMamba backbone. It integrates two lightweight, plug-and-play modules: (1) Bi-Temporal Deformable Alignment (BTDA), which explicitly introduces geometric awareness to correct spatial misalignments at the semantic feature level; and (2) a Scale-Sparse Change Amplifier(SSCA), which uses multi-source cues to selectively amplify high-confidence change signals while suppressing noise before the final classification. This synergistic design first establishes geometric consistency with BTDA to reduce pseudo-changes, then leverages SSCA to sharpen boundaries and enhance the visibility of small or subtle targets. Experiments show our method significantly improves performance over the strong ChangeMamba baseline, increasing the F1-score from 0.5730 to 0.5903 and IoU from 0.4015 to 0.4187. The results confirm the effectiveness of our “align-then-enhance” strategy, offering a robust and easily deployable solution that transparently addresses both geometric and feature-level challenges in RSCD.

Method: Developed EyePCR benchmark with 210k+ VQAs covering 1048 fine-grained attributes for perception, 25k+ medical knowledge graph triplets for comprehension, and four clinically grounded reasoning tasks. Created EyePCR-MLLM as a domain-adapted variant of Qwen2.5-VL-7B.

Result: EyePCR-MLLM achieves highest accuracy on MCQ perception tasks among compared models, outperforms open-source models in comprehension and reasoning, and rivals commercial models like GPT-4.1.

Conclusion: EyePCR reveals limitations of existing MLLMs in surgical cognition and provides foundation for benchmarking and enhancing clinical reliability of surgical video understanding models.

Abstract: MLLMs (Multimodal Large Language Models) have showcased remarkable capabilities, but their performance in high-stakes, domain-specific scenarios like surgical settings, remains largely under-explored. To address this gap, we develop \textbf{EyePCR}, a large-scale benchmark for ophthalmic surgery analysis, grounded in structured clinical knowledge to evaluate cognition across \textit{Perception}, \textit{Comprehension} and \textit{Reasoning}. EyePCR offers a richly annotated corpus with more than 210k VQAs, which cover 1048 fine-grained attributes for multi-view perception, medical knowledge graph of more than 25k triplets for comprehension, and four clinically grounded reasoning tasks. The rich annotations facilitate in-depth cognitive analysis, simulating how surgeons perceive visual cues and combine them with domain knowledge to make decisions, thus greatly improving models’ cognitive ability. In particular, \textbf{EyePCR-MLLM}, a domain-adapted variant of Qwen2.5-VL-7B, achieves the highest accuracy on MCQs for \textit{Perception} among compared models and outperforms open-source models in \textit{Comprehension} and \textit{Reasoning}, rivalling commercial models like GPT-4.1. EyePCR reveals the limitations of existing MLLMs in surgical cognition and lays the foundation for benchmarking and enhancing clinical reliability of surgical video understanding models.

Method: Created TennisTV benchmark by modeling tennis rallies as temporal-ordered sequences of stroke events using automated pipelines for filtering and question generation, covering 8 tasks at rally and stroke levels with 2,500 human-verified questions.

Result: Evaluation of 16 representative MLLMs revealed substantial shortcomings in tennis video understanding, showing that current models perform poorly on this challenging task.

Conclusion: Two key insights emerged: (i) frame-sampling density should be tailored and balanced across different tasks, and (ii) improving temporal grounding is essential for stronger reasoning capabilities in sports video understanding.

Abstract: Multimodal large language models (MLLMs) excel at general video understanding but struggle with fast, high-frequency sports like tennis, where rally clips are short yet information-dense. To systematically evaluate MLLMs in this challenging domain, we present TennisTV, the first and most comprehensive benchmark for tennis video understanding. TennisTV models each rally as a temporal-ordered sequence of consecutive stroke events, using automated pipelines for filtering and question generation. It covers 8 tasks at rally and stroke levels and includes 2,500 human-verified questions. Evaluating 16 representative MLLMs, we provide the first systematic assessment of tennis video understanding. Results reveal substantial shortcomings and yield two key insights: (i) frame-sampling density should be tailored and balanced across tasks, and (ii) improving temporal grounding is essential for stronger reasoning.

Method: Uses LLMs to extract fine-grained WSI-relevant descriptions from pathology reports, implements self-distillation to filter irrelevant WSI features guided by textual knowledge, and employs risk-aware mix-up strategy to enhance training data quantity and diversity.

Result: Extensive experiments on CRC and TCGA-BRCA datasets demonstrate Rasa’s superior effectiveness compared to state-of-the-art methods.

Conclusion: The proposed Rasa framework successfully integrates pathology reports with WSI analysis through self-distillation, providing a more effective approach for cancer prognosis prediction by leveraging both visual and textual patient information.

Abstract: Survival analysis based on Whole Slide Images (WSIs) is crucial for evaluating cancer prognosis, as they offer detailed microscopic information essential for predicting patient outcomes. However, traditional WSI-based survival analysis usually faces noisy features and limited data accessibility, hindering their ability to capture critical prognostic features effectively. Although pathology reports provide rich patient-specific information that could assist analysis, their potential to enhance WSI-based survival analysis remains largely unexplored. To this end, this paper proposes a novel Report-auxiliary self-distillation (Rasa) framework for WSI-based survival analysis. First, advanced large language models (LLMs) are utilized to extract fine-grained, WSI-relevant textual descriptions from original noisy pathology reports via a carefully designed task prompt. Next, a self-distillation-based pipeline is designed to filter out irrelevant or redundant WSI features for the student model under the guidance of the teacher model’s textual knowledge. Finally, a risk-aware mix-up strategy is incorporated during the training of the student model to enhance both the quantity and diversity of the training data. Extensive experiments carried out on our collected data (CRC) and public data (TCGA-BRCA) demonstrate the superior effectiveness of Rasa against state-of-the-art methods. Our code is available at https://github.com/zhengwang9/Rasa.

Method: Proposes Pseudo-label Consistency-Guided Sample Refinement (PCSR) with confidence-based estimation to distinguish clean/noisy pairs, pseudo-label consistency to refine noisy pairs, and PCS score to separate ambiguous/refinable samples. Uses Adaptive Pair Optimization with robust loss for ambiguous samples and text replacement for refinable ones.

Result: Extensive experiments on CC152K, MS-COCO and Flickr30K demonstrate improved retrieval robustness under noisy supervision compared to existing methods.

Conclusion: PCSR effectively handles noisy correspondences by leveraging pseudo-label consistency and adaptive optimization, significantly enhancing cross-modal retrieval performance in real-world noisy scenarios.

Abstract: Cross-modal retrieval aims to align different modalities via semantic similarity. However, existing methods often assume that image-text pairs are perfectly aligned, overlooking Noisy Correspondences in real data. These misaligned pairs misguide similarity learning and degrade retrieval performance. Previous methods often rely on coarse-grained categorizations that simply divide data into clean and noisy samples, overlooking the intrinsic diversity within noisy instances. Moreover, they typically apply uniform training strategies regardless of sample characteristics, resulting in suboptimal sample utilization for model optimization. To address the above challenges, we introduce a novel framework, called Pseudo-label Consistency-Guided Sample Refinement (PCSR), which enhances correspondence reliability by explicitly dividing samples based on pseudo-label consistency. Specifically, we first employ a confidence-based estimation to distinguish clean and noisy pairs, then refine the noisy pairs via pseudo-label consistency to uncover structurally distinct subsets. We further proposed a Pseudo-label Consistency Score (PCS) to quantify prediction stability, enabling the separation of ambiguous and refinable samples within noisy pairs. Accordingly, we adopt Adaptive Pair Optimization (APO), where ambiguous samples are optimized with robust loss functions and refinable ones are enhanced via text replacement during training. Extensive experiments on CC152K, MS-COCO and Flickr30K validate the effectiveness of our method in improving retrieval robustness under noisy supervision.

Method: Two key innovations: (1) personalized strategy aggregating only global parameters while retaining L-MoE for domain-specific features; (2) decoupled global-local fine-tuning using teacher-student knowledge distillation to bridge global and local models.

Result: Extensive experiments on two public datasets show significant improvement in segmentation performance, robust cross-domain adaptation, and reduced communication overhead.

Conclusion: The proposed framework successfully addresses the challenges of using SAM in federated learning for medical image segmentation, achieving better performance while maintaining privacy and efficiency.

Abstract: Medical image segmentation is crucial for computer-aided diagnosis, yet privacy constraints hinder data sharing across institutions. Federated learning addresses this limitation, but existing approaches often rely on lightweight architectures that struggle with complex, heterogeneous data. Recently, the Segment Anything Model (SAM) has shown outstanding segmentation capabilities; however, its massive encoder poses significant challenges in federated settings. In this work, we present the first personalized federated SAM framework tailored for heterogeneous data scenarios in medical image segmentation. Our framework integrates two key innovations: (1) a personalized strategy that aggregates only the global parameters to capture cross-client commonalities while retaining the designed L-MoE (Localized Mixture-of-Experts) component to preserve domain-specific features; and (2) a decoupled global-local fine-tuning mechanism that leverages a teacher-student paradigm via knowledge distillation to bridge the gap between the global shared model and the personalized local models, thereby mitigating overgeneralization. Extensive experiments on two public datasets validate that our approach significantly improves segmentation performance, achieves robust cross-domain adaptation, and reduces communication overhead.

Method: Proposes UNIV with two key innovations: 1) Patch-wise Cross-modality Contrastive Learning (PCCL) for attention-guided cross-modal feature alignment, and 2) dual-knowledge preservation mechanism using LoRA adapters (2% parameters) with synchronous distillation to prevent catastrophic forgetting. Introduces MVIP dataset with 98,992 aligned visible-infrared image pairs.

Result: Superior performance on infrared tasks (+1.7 mIoU in semantic segmentation and +0.7 mAP in object detection) while maintaining 99%+ of baseline performance on visible RGB tasks.

Conclusion: UNIV successfully bridges the performance gap between RGB and infrared modalities through biologically inspired mechanisms, providing a robust foundation model for multimodal perception applications.

Abstract: The demand for joint RGB-visible and infrared perception is growing rapidly, particularly to achieve robust performance under diverse weather conditions. Although pre-trained models for RGB-visible and infrared data excel in their respective domains, they often underperform in multimodal scenarios, such as autonomous vehicles equipped with both sensors. To address this challenge, we propose a biologically inspired UNified foundation model for Infrared and Visible modalities (UNIV), featuring two key innovations. First, we introduce Patch-wise Cross-modality Contrastive Learning (PCCL), an attention-guided distillation framework that mimics retinal horizontal cells’ lateral inhibition, which enables effective cross-modal feature alignment while remaining compatible with any transformer-based architecture. Second, our dual-knowledge preservation mechanism emulates the retina’s bipolar cell signal routing - combining LoRA adapters (2% added parameters) with synchronous distillation to prevent catastrophic forgetting, thereby replicating the retina’s photopic (cone-driven) and scotopic (rod-driven) functionality. To support cross-modal learning, we introduce the MVIP dataset, the most comprehensive visible-infrared benchmark to date. It contains 98,992 precisely aligned image pairs spanning diverse scenarios. Extensive experiments demonstrate UNIV’s superior performance on infrared tasks (+1.7 mIoU in semantic segmentation and +0.7 mAP in object detection) while maintaining 99%+ of the baseline performance on visible RGB tasks. Our code is available at https://github.com/fangyuanmao/UNIV.

Method: GS-Scale stores Gaussians in host memory and transfers subsets to GPU on demand. It uses three optimizations: selective offloading for fast frustum culling, parameter forwarding to pipeline CPU optimizer updates, and deferred optimizer update to minimize memory accesses for zero-gradient Gaussians.

Result: GS-Scale reduces GPU memory demands by 3.3-5.6x, scales from 4M to 18M Gaussians on RTX 4070 Mobile GPU, and achieves 23-35% LPIPS improvement.

Conclusion: GS-Scale enables efficient large-scale 3D Gaussian Splatting training on consumer-grade GPUs by significantly reducing memory requirements while maintaining training performance.

Abstract: The advent of 3D Gaussian Splatting has revolutionized graphics rendering by delivering high visual quality and fast rendering speeds. However, training large-scale scenes at high quality remains challenging due to the substantial memory demands required to store parameters, gradients, and optimizer states, which can quickly overwhelm GPU memory. To address these limitations, we propose GS-Scale, a fast and memory-efficient training system for 3D Gaussian Splatting. GS-Scale stores all Gaussians in host memory, transferring only a subset to the GPU on demand for each forward and backward pass. While this dramatically reduces GPU memory usage, it requires frustum culling and optimizer updates to be executed on the CPU, introducing slowdowns due to CPU’s limited compute and memory bandwidth. To mitigate this, GS-Scale employs three system-level optimizations: (1) selective offloading of geometric parameters for fast frustum culling, (2) parameter forwarding to pipeline CPU optimizer updates with GPU computation, and (3) deferred optimizer update to minimize unnecessary memory accesses for Gaussians with zero gradients. Our extensive evaluations on large-scale datasets demonstrate that GS-Scale significantly lowers GPU memory demands by 3.3-5.6x, while achieving training speeds comparable to GPU without host offloading. This enables large-scale 3D Gaussian Splatting training on consumer-grade GPUs; for instance, GS-Scale can scale the number of Gaussians from 4 million to 18 million on an RTX 4070 Mobile GPU, leading to 23-35% LPIPS (learned perceptual image patch similarity) improvement.

Method: Two-stage process: (1) extract peripheral features and generate sample map, (2) sample fixation patches using non-maximum suppression and encode with weight-shared encoder, then fuse representations using contextualized selective attention.

Result: Achieves comparable performance to state-of-the-art approaches on standard FGVC benchmarks (CUB-200-2011, NABirds, Food-101, Stanford-Dogs) and challenging insect datasets (EU-Moths, Ecuador-Moths, AMI-Moths), consistently outperforming baseline encoder.

Conclusion: The proposed saccadic vision-inspired approach effectively addresses spatial redundancy in part-based FGVC methods while maintaining competitive performance across diverse datasets.

Abstract: Fine-grained visual classification (FGVC) requires distinguishing between visually similar categories through subtle, localized features - a task that remains challenging due to high intra-class variability and limited inter-class differences. Existing part-based methods often rely on complex localization networks that learn mappings from pixel to sample space, requiring a deep understanding of image content while limiting feature utility for downstream tasks. In addition, sampled points frequently suffer from high spatial redundancy, making it difficult to quantify the optimal number of required parts. Inspired by human saccadic vision, we propose a two-stage process that first extracts peripheral features (coarse view) and generates a sample map, from which fixation patches are sampled and encoded in parallel using a weight-shared encoder. We employ contextualized selective attention to weigh the impact of each fixation patch before fusing peripheral and focus representations. To prevent spatial collapse - a common issue in part-based methods - we utilize non-maximum suppression during fixation sampling to eliminate redundancy. Comprehensive evaluation on standard FGVC benchmarks (CUB-200-2011, NABirds, Food-101 and Stanford-Dogs) and challenging insect datasets (EU-Moths, Ecuador-Moths and AMI-Moths) demonstrates that our method achieves comparable performance to state-of-the-art approaches while consistently outperforming our baseline encoder.

Method: A framework integrating 3D Gaussian Splatting for contactless fingerprint 3D registration, reconstruction, and generation from sparse input images without camera parameters.

Result: Experiments show accurate 3D fingerprint alignment and reconstruction from 2D images, with high-quality contactless fingerprint generation that improves recognition performance.

Conclusion: This work presents the first application of 3D Gaussian Splatting to fingerprint recognition, offering a new paradigm that enhances contactless fingerprint recognition through 3D reconstruction and generation.

Abstract: Researchers have conducted many pioneer researches on contactless fingerprints, yet the performance of contactless fingerprint recognition still lags behind contact-based methods primary due to the insufficient contactless fingerprint data with pose variations and lack of the usage of implicit 3D fingerprint representations. In this paper, we introduce a novel contactless fingerprint 3D registration, reconstruction and generation framework by integrating 3D Gaussian Splatting, with the goal of offering a new paradigm for contactless fingerprint recognition that integrates 3D fingerprint reconstruction and generation. To our knowledge, this is the first work to apply 3D Gaussian Splatting to the field of fingerprint recognition, and the first to achieve effective 3D registration and complete reconstruction of contactless fingerprints with sparse input images and without requiring camera parameters information. Experiments on 3D fingerprint registration, reconstruction, and generation prove that our method can accurately align and reconstruct 3D fingerprints from 2D images, and sequentially generates high-quality contactless fingerprints from 3D model, thus increasing the performances for contactless fingerprint recognition.

Method: Uses synchronized multimodal satellite data (VIS/TIR imagery from geostationary satellites and vertical cloud phase profiles from CALIOP/CALIPSO lidar and CPR/CloudSat radar) with supervised learning. Compares SGMAGNet against UNet variants and SegNet for multi-scale spatial pattern capture.

Result: SGMAGNet achieves superior performance with Precision: 0.922, Recall: 0.858, F1-score: 0.763, and IoU: 0.617, significantly outperforming all baseline models, especially in complex multi-layer and boundary transition regions.

Conclusion: The proposed framework successfully transforms multimodal satellite observations into detailed 3D cloud phase structures, demonstrating SGMAGNet’s effectiveness for operational cloud phase profile retrieval and potential integration with NWP systems.

Abstract: Cloud phase profiles are critical for numerical weather prediction (NWP), as they directly affect radiative transfer and precipitation processes. In this study, we present a benchmark dataset and a baseline framework for transforming multimodal satellite observations into detailed 3D cloud phase structures, aiming toward operational cloud phase profile retrieval and future integration with NWP systems to improve cloud microphysics parameterization. The multimodal observations consist of (1) high–spatiotemporal–resolution, multi-band visible (VIS) and thermal infrared (TIR) imagery from geostationary satellites, and (2) accurate vertical cloud phase profiles from spaceborne lidar (CALIOP\slash CALIPSO) and radar (CPR\slash CloudSat). The dataset consists of synchronized image–profile pairs across diverse cloud regimes, defining a supervised learning task: given VIS/TIR patches, predict the corresponding 3D cloud phase structure. We adopt SGMAGNet as the main model and compare it with several baseline architectures, including UNet variants and SegNet, all designed to capture multi-scale spatial patterns. Model performance is evaluated using standard classification metrics, including Precision, Recall, F1-score, and IoU. The results demonstrate that SGMAGNet achieves superior performance in cloud phase reconstruction, particularly in complex multi-layer and boundary transition regions. Quantitatively, SGMAGNet attains a Precision of 0.922, Recall of 0.858, F1-score of 0.763, and an IoU of 0.617, significantly outperforming all baselines across these key metrics.

Method: Uses PCA to estimate surface normals from available data, then employs these normals as regularizers in a reconstruction model solved with an operator-splitting method.

Result: The model successfully infers surface structures in missing regions and reconstructs underlying surfaces, outperforming existing methods.

Conclusion: The PCA-based approach with normal estimation regularization effectively handles incomplete point cloud data for surface reconstruction.

Abstract: Point cloud data represents a crucial category of information for mathematical modeling, and surface reconstruction from such data is an important task across various disciplines. However, during the scanning process, the collected point cloud data may fail to cover the entire surface due to factors such as high light-absorption rate and occlusions, resulting in incomplete datasets. Inferring surface structures in data-missing regions and successfully reconstructing the surface poses a challenge. In this paper, we present a Principal Component Analysis (PCA) based model for surface reconstruction from incomplete point cloud data. Initially, we employ PCA to estimate the normal information of the underlying surface from the available point cloud data. This estimated normal information serves as a regularizer in our model, guiding the reconstruction of the surface, particularly in areas with missing data. Additionally, we introduce an operator-splitting method to effectively solve the proposed model. Through systematic experimentation, we demonstrate that our model successfully infers surface structures in data-missing regions and well reconstructs the underlying surfaces, outperforming existing methodologies.

Method: Model each camera as a 3D Gaussian (camera splat), place virtual point cameras near surfaces to observe camera splat distributions, and continuously refine camera splats differentiably to achieve target distributions.

Result: The optimized views demonstrate superior performance compared to Farthest View Sampling in capturing complex view-dependent effects including metallic reflections and detailed textures.

Conclusion: Camera Splatting provides an effective framework for novel view synthesis that outperforms FVS in handling challenging view-dependent phenomena through differentiable camera splat optimization.

Abstract: We propose Camera Splatting, a novel view optimization framework for novel view synthesis. Each camera is modeled as a 3D Gaussian, referred to as a camera splat, and virtual cameras, termed point cameras, are placed at 3D points sampled near the surface to observe the distribution of camera splats. View optimization is achieved by continuously and differentiably refining the camera splats so that desirable target distributions are observed from the point cameras, in a manner similar to the original 3D Gaussian splatting. Compared to the Farthest View Sampling (FVS) approach, our optimized views demonstrate superior performance in capturing complex view-dependent phenomena, including intense metallic reflections and intricate textures such as text.

Method: Uses grid-based filtering, adaptive resolution projection, and image enhancement to create top-down density maps. Integrates SAM’s zero-shot learning for room segmentation with adaptive prompt points and multistage filtering, followed by joint mask and point cloud analysis for contour extraction and regularization.

Result: Tests on Giblayout and ISPRS datasets show superior accuracy, recall, and robustness compared to traditional methods, particularly in noisy and complex settings.

Conclusion: FloorSAM effectively reconstructs accurate floor plans and recovers room topological relationships, demonstrating better performance than existing geometric algorithms and deep learning approaches.

Abstract: Reconstructing building floor plans from point cloud data is key for indoor navigation, BIM, and precise measurements. Traditional methods like geometric algorithms and Mask R-CNN-based deep learning often face issues with noise, limited generalization, and loss of geometric details. We propose FloorSAM, a framework that integrates point cloud density maps with the Segment Anything Model (SAM) for accurate floor plan reconstruction from LiDAR data. Using grid-based filtering, adaptive resolution projection, and image enhancement, we create robust top-down density maps. FloorSAM uses SAM’s zero-shot learning for precise room segmentation, improving reconstruction across diverse layouts. Room masks are generated via adaptive prompt points and multistage filtering, followed by joint mask and point cloud analysis for contour extraction and regularization. This produces accurate floor plans and recovers room topological relationships. Tests on Giblayout and ISPRS datasets show better accuracy, recall, and robustness than traditional methods, especially in noisy and complex settings. Code and materials: github.com/Silentbarber/FloorSAM.

Method: Proposes multi-scale attention features for local/global style imitation, includes word layout for spacing, and uses a conditional diffusion model to predict strokes guided by calligraphic style and word layout.

Result: The proposed diffusion model outperforms current state-of-the-art stroke generation methods and is competitive with recent image generation networks.

Conclusion: Incorporating word layout and using conditional diffusion for stroke generation enables better handwriting imitation and calligraphic style reproduction compared to previous approaches.

Abstract: Handwriting stroke generation is crucial for improving the performance of tasks such as handwriting recognition and writers order recovery. In handwriting stroke generation, it is significantly important to imitate the sample calligraphic style. The previous studies have suggested utilizing the calligraphic features of the handwriting. However, they had not considered word spacing (word layout) as an explicit handwriting feature, which results in inconsistent word spacing for style imitation. Firstly, this work proposes multi-scale attention features for calligraphic style imitation. These multi-scale feature embeddings highlight the local and global style features. Secondly, we propose to include the words layout, which facilitates word spacing for handwriting stroke generation. Moreover, we propose a conditional diffusion model to predict strokes in contrast to previous work, which directly generated style images. Stroke generation provides additional temporal coordinate information, which is lacking in image generation. Hence, our proposed conditional diffusion model for stroke generation is guided by calligraphic style and word layout for better handwriting imitation and stroke generation in a calligraphic style. Our experimentation shows that the proposed diffusion model outperforms the current state-of-the-art stroke generation and is competitive with recent image generation networks.

Method: Created ORIC benchmark using LLM-guided sampling for contextually incongruous objects and CLIP-guided sampling for plausible but nonexistent objects likely to be hallucinated.

Result: Evaluation of 18 LVLMs and 2 open-vocabulary detection models showed significant recognition gaps in incongruous contexts.

Conclusion: The work highlights LVLMs’ limitations in context-aware object recognition and encourages further research in this area.

Abstract: Large Vision-Language Models (LVLMs) have made significant strides in image caption, visual question answering, and robotics by integrating visual and textual information. However, they remain prone to errors in incongruous contexts, where objects appear unexpectedly or are absent when contextually expected. This leads to two key recognition failures: object misidentification and hallucination. To systematically examine this issue, we introduce the Object Recognition in Incongruous Context Benchmark (ORIC), a novel benchmark that evaluates LVLMs in scenarios where object-context relationships deviate from expectations. ORIC employs two key strategies: (1) LLM-guided sampling, which identifies objects that are present but contextually incongruous, and (2) CLIP-guided sampling, which detects plausible yet nonexistent objects that are likely to be hallucinated, thereby creating an incongruous context. Evaluating 18 LVLMs and two open-vocabulary detection models, our results reveal significant recognition gaps, underscoring the challenges posed by contextual incongruity. This work provides critical insights into LVLMs’ limitations and encourages further research on context-aware object recognition.

Method: SegReg decomposes input images into anatomically coherent subregions through segmentation, processes these localized domains with the same registration backbone to compute optimized partial deformation fields, then integrates them into a global deformation field.

Result: Achieves near-perfect structural alignment (98.23% Dice on critical anatomies) with ground-truth segmentation, and outperforms existing methods by 2-12% across three clinical registration scenarios (cardiac, abdominal, and lung images) even with automatic segmentation.

Conclusion: SegReg demonstrates a near-linear dependence of registration accuracy on segmentation quality, transforming the registration challenge into a segmentation problem.

Abstract: Deep learning has revolutionized image registration by its ability to handle diverse tasks while achieving significant speed advantages over conventional approaches. Current approaches, however, often employ globally uniform smoothness constraints that fail to accommodate the complex, regionally varying deformations characteristic of anatomical motion. To address this limitation, we propose SegReg, a Segmentation-driven Registration framework that implements anatomically adaptive regularization by exploiting region-specific deformation patterns. Our SegReg first decomposes input moving and fixed images into anatomically coherent subregions through segmentation. These localized domains are then processed by the same registration backbone to compute optimized partial deformation fields, which are subsequently integrated into a global deformation field. SegReg achieves near-perfect structural alignment (98.23% Dice on critical anatomies) using ground-truth segmentation, and outperforms existing methods by 2-12% across three clinical registration scenarios (cardiac, abdominal, and lung images) even with automatic segmentation. Our SegReg demonstrates a near-linear dependence of registration accuracy on segmentation quality, transforming the registration challenge into a segmentation problem. The source code will be released upon manuscript acceptance.

Method: Proposes PTP that integrates bottom-up visual saliency at region/token levels with top-down instruction-guided importance, inspired by human visual attention mechanisms.

Result: Extensive experiments across 13 benchmarks show substantial reduction in computational overhead and inference latency with minimal performance loss.

Conclusion: PTP effectively addresses computational inefficiency in LVLMs for high-resolution images through selective token pruning based on visual saliency and task relevance.

Abstract: Large Vision-Language Models (LVLMs) have significantly advanced multimodal understanding but still struggle with efficiently processing high-resolution images. Recent approaches partition high-resolution images into multiple sub-images, dramatically increasing the number of visual tokens and causing exponential computational overhead during inference. To address these limitations, we propose a training-free token pruning strategy, Pyramid Token Pruning (PTP), that integrates bottom-up visual saliency at both region and token levels with top-down instruction-guided importance. Inspired by human visual attention mechanisms, PTP selectively retains more tokens from visually salient regions and further leverages textual instructions to pinpoint tokens most relevant to specific multimodal tasks. Extensive experiments across 13 diverse benchmarks demonstrate that our method substantially reduces computational overhead and inference latency with minimal performance loss.

Method: Proposes Continual Backpropagation Prompt Network (CBPNet) with an Efficient CBP Block that adaptively reinitializes underutilized parameters to restore learning vitality.

Result: Achieves 69.41% accuracy on Split ImageNet-R (state-of-the-art) and improves average accuracy by over 1% on Split CIFAR-100, using less than 0.2% additional parameters of backbone size.

Conclusion: CBPNet effectively restores plasticity in continual learning while maintaining parameter efficiency, making it suitable for edge devices requiring real-time responses.

Abstract: To meet the demands of applications like robotics and autonomous driving that require real-time responses to dynamic environments, efficient continual learning methods suitable for edge devices have attracted increasing attention. In this transition, using frozen pretrained models with prompts has become a mainstream strategy to combat catastrophic forgetting. However, this approach introduces a new critical bottleneck: plasticity loss, where the model’s ability to learn new knowledge diminishes due to the frozen backbone and the limited capacity of prompt parameters. We argue that the reduction in plasticity stems from a lack of update vitality in underutilized parameters during the training process. To this end, we propose the Continual Backpropagation Prompt Network (CBPNet), an effective and parameter efficient framework designed to restore the model’s learning vitality. We innovatively integrate an Efficient CBP Block that counteracts plasticity decay by adaptively reinitializing these underutilized parameters. Experimental results on edge devices demonstrate CBPNet’s effectiveness across multiple benchmarks. On Split CIFAR-100, it improves average accuracy by over 1% against a strong baseline, and on the more challenging Split ImageNet-R, it achieves a state of the art accuracy of 69.41%. This is accomplished by training additional parameters that constitute less than 0.2% of the backbone’s size, validating our approach.

Method: Introduces MedForensics dataset with six medical modalities and twelve generative models, and proposes DSKI detector with cross-domain fine-trace adapter (CDFA) for spatial/noise domain feature extraction and medical forensic retrieval module (MFRM) for few-shot retrieval.

Result: DSKI significantly outperforms both existing methods and human experts, achieving superior accuracy across multiple medical modalities.

Conclusion: The proposed MedForensics dataset and DSKI detector effectively address the critical need for specialized medical image forensics, demonstrating state-of-the-art performance in detecting AI-generated medical images.

Abstract: The rapid advancement of generative AI in medical imaging has introduced both significant opportunities and serious challenges, especially the risk that fake medical images could undermine healthcare systems. These synthetic images pose serious risks, such as diagnostic deception, financial fraud, and misinformation. However, research on medical forensics to counter these threats remains limited, and there is a critical lack of comprehensive datasets specifically tailored for this field. Additionally, existing media forensic methods, which are primarily designed for natural or facial images, are inadequate for capturing the distinct characteristics and subtle artifacts of AI-generated medical images. To tackle these challenges, we introduce \textbf{MedForensics}, a large-scale medical forensics dataset encompassing six medical modalities and twelve state-of-the-art medical generative models. We also propose \textbf{DSKI}, a novel \textbf{D}ual-\textbf{S}tage \textbf{K}nowledge \textbf{I}nfusing detector that constructs a vision-language feature space tailored for the detection of AI-generated medical images. DSKI comprises two core components: 1) a cross-domain fine-trace adapter (CDFA) for extracting subtle forgery clues from both spatial and noise domains during training, and 2) a medical forensic retrieval module (MFRM) that boosts detection accuracy through few-shot retrieval during testing. Experimental results demonstrate that DSKI significantly outperforms both existing methods and human experts, achieving superior accuracy across multiple medical modalities.

Method: TrueMoE uses a Discriminative Expert Array organized along manifold structure and perceptual granularity axes, with dual-routing mechanism (granularity-aware sparse router and manifold-aware dense router) to adaptively assign inputs to relevant experts.

Result: Extensive experiments show TrueMoE achieves superior generalization and robustness across a wide spectrum of generative models.

Conclusion: The collaborative inference across multiple specialized discriminative subspaces provides better detection performance than unified approaches, making TrueMoE an effective framework for synthetic image detection.

Abstract: The rapid progress of generative models has made synthetic image detection an increasingly critical task. Most existing approaches attempt to construct a single, universal discriminative space to separate real from fake content. However, such unified spaces tend to be complex and brittle, often struggling to generalize to unseen generative patterns. In this work, we propose TrueMoE, a novel dual-routing Mixture-of-Discriminative-Experts framework that reformulates the detection task as a collaborative inference across multiple specialized and lightweight discriminative subspaces. At the core of TrueMoE is a Discriminative Expert Array (DEA) organized along complementary axes of manifold structure and perceptual granularity, enabling diverse forgery cues to be captured across subspaces. A dual-routing mechanism, comprising a granularity-aware sparse router and a manifold-aware dense router, adaptively assigns input images to the most relevant experts. Extensive experiments across a wide spectrum of generative models demonstrate that TrueMoE achieves superior generalization and robustness.

Method: Uses tensor decomposition and reconstruction of vision encoder representations to filter adversarial noise while preserving semantic meaning. Specifically employs Tensor Train decomposition with low rank (8-32) and low residual strength (α=0.1-0.2).

Result: On Flickr30K, the method restores 12.3% performance lost to attacks, improving Recall@1 accuracy from 7.5% to 19.8%. On COCO, it recovers 8.1% performance, increasing accuracy from 3.8% to 11.9%.

Conclusion: The proposed tensor decomposition method provides a practical, plug-and-play defense solution with minimal overhead for existing pre-trained vision language models.

Abstract: Vision language models (VLMs) excel in multimodal understanding but are prone to adversarial attacks. Existing defenses often demand costly retraining or significant architecture changes. We introduce a lightweight defense using tensor decomposition suitable for any pre-trained VLM, requiring no retraining. By decomposing and reconstructing vision encoder representations, it filters adversarial noise while preserving meaning. Experiments with CLIP on COCO and Flickr30K show improved robustness. On Flickr30K, it restores 12.3% performance lost to attacks, raising Recall@1 accuracy from 7.5% to 19.8%. On COCO, it recovers 8.1% performance, improving accuracy from 3.8% to 11.9%. Analysis shows Tensor Train decomposition with low rank (8-32) and low residual strength ($\alpha=0.1-0.2$) is optimal. This method is a practical, plug-and-play solution with minimal overhead for existing VLMs.

Method: Combines Temporal Apex Distillation (TAD) for keyframe selection via variation scoring, inflection detection, and prioritized distillation, and KF-GRPO using contrastive learning with saliency-enhanced rewards to leverage both frame content and temporal relationships.

Result: Achieves 69.1% on VideoMME and 52.7% on LVBench, surpassing previous approaches while enabling a 7B parameter model to match performance of 72B parameter alternatives.

Conclusion: The proposed framework effectively addresses computational challenges in video understanding through intelligent keyframe selection and temporal reasoning, achieving superior performance with significantly smaller models.

Abstract: Current state-of-the-art video understanding methods typically struggle with two critical challenges: (1) the computational infeasibility of processing every frame in dense video content and (2) the difficulty in identifying semantically significant frames through naive uniform sampling strategies. In this paper, we propose a novel video understanding framework, called ChronoForge-RL, which combines Temporal Apex Distillation (TAD) and KeyFrame-aware Group Relative Policy Optimization (KF-GRPO) to tackle these issues. Concretely, we introduce a differentiable keyframe selection mechanism that systematically identifies semantic inflection points through a three-stage process to enhance computational efficiency while preserving temporal information. Then, two particular modules are proposed to enable effective temporal reasoning: Firstly, TAD leverages variation scoring, inflection detection, and prioritized distillation to select the most informative frames. Secondly, we introduce KF-GRPO which implements a contrastive learning paradigm with a saliency-enhanced reward mechanism that explicitly incentivizes models to leverage both frame content and temporal relationships. Finally, our proposed ChronoForge-RL achieves 69.1% on VideoMME and 52.7% on LVBench compared to baseline methods, clearly surpassing previous approaches while enabling our 7B parameter model to achieve performance comparable to 72B parameter alternatives.

Method: Derives (i) infinitesimal relationships using semi-groups and Lie groups concepts, and (ii) macroscopic cascade smoothing properties describing how coarser-scale receptive field responses can be computed from finer-scale outputs.

Result: Provides understanding of spatial and spatio-temporal receptive field relationships across filter parameters, enabling more efficient computation schemes and theoretical models for biological vision.

Conclusion: The derived relationships offer tools for designing efficient receptive field computation methods and formulating idealized models of simple cell computations in biological vision systems.

Abstract: Because of the variabilities of real-world image structures under the natural image transformations that arise when observing similar objects or spatio-temporal events under different viewing conditions, the receptive field responses computed in the earliest layers of the visual hierarchy may be strongly influenced by such geometric image transformations. One way of handling this variability is by basing the vision system on covariant receptive field families, which expand the receptive field shapes over the degrees of freedom in the image transformations. This paper addresses the problem of deriving relationships between spatial and spatio-temporal receptive field responses obtained for different values of the shape parameters in the resulting multi-parameter families of receptive fields. For this purpose, we derive both (i) infinitesimal relationships, roughly corresponding to a combination of notions from semi-groups and Lie groups, as well as (ii) macroscopic cascade smoothing properties, which describe how receptive field responses at coarser spatial and temporal scales can be computed by applying smaller support incremental filters to the output from corresponding receptive fields at finer spatial and temporal scales, structurally related to the notion of Lie algebras, although with directional preferences. The presented results provide (i) a deeper understanding of the relationships between spatial and spatio-temporal receptive field responses for different values of the filter parameters, which can be used for both (ii) designing more efficient schemes for computing receptive field responses over populations of multi-parameter families of receptive fields, as well as (iii)~formulating idealized theoretical models of the computations of simple cells in biological vision.

Method: CIDER uses a lightweight detector to identify branded content and a Vision-Language Model (VLM) to generate stylistically divergent alternatives through prompt refinement at inference-time.

Result: Extensive experiments show CIDER significantly reduces both explicit and implicit brand biases while maintaining image quality and aesthetic appeal, as measured by the proposed Brand Neutrality Score (BNS).

Conclusion: CIDER offers a practical solution for more original and equitable content generation, contributing to the development of trustworthy generative AI without requiring model retraining.

Abstract: Text-to-image (T2I) models exhibit a significant yet under-explored “brand bias”, a tendency to generate contents featuring dominant commercial brands from generic prompts, posing ethical and legal risks. We propose CIDER, a novel, model-agnostic framework to mitigate bias at inference-time through prompt refinement to avoid costly retraining. CIDER uses a lightweight detector to identify branded content and a Vision-Language Model (VLM) to generate stylistically divergent alternatives. We introduce the Brand Neutrality Score (BNS) to quantify this issue and perform extensive experiments on leading T2I models. Results show CIDER significantly reduces both explicit and implicit biases while maintaining image quality and aesthetic appeal. Our work offers a practical solution for more original and equitable content, contributing to the development of trustworthy generative AI.

Method: Used egocentric videos with foveation models to create sequences where visual content becomes less distinct towards periphery. Trained two bio-inspired self-supervised models with time-based learning objectives.

Result: Foveated vision modeling improved quality of learned object representations by making objects appear bigger and creating better trade-off between central and peripheral information.

Conclusion: Incorporating foveated vision aspects makes models of human visual learning more realistic and performant, representing an important step forward.

Abstract: Recent self-supervised learning models simulate the development of semantic object representations by training on visual experience similar to that of toddlers. However, these models ignore the foveated nature of human vision with high/low resolution in the center/periphery of the visual field. Here, we investigate the role of this varying resolution in the development of object representations. We leverage two datasets of egocentric videos that capture the visual experience of humans during interactions with objects. We apply models of human foveation and cortical magnification to modify these inputs, such that the visual content becomes less distinct towards the periphery. The resulting sequences are used to train two bio-inspired self-supervised learning models that implement a time-based learning objective. Our results show that modeling aspects of foveated vision improves the quality of the learned object representations in this setting. Our analysis suggests that this improvement comes from making objects appear bigger and inducing a better trade-off between central and peripheral visual information. Overall, this work takes a step towards making models of humans’ learning of visual representations more realistic and performant.

Method: Uses a shared vision encoder with two lightweight adapters producing continuous embeddings for understanding and discrete tokens for generation. Combines an autoregressive LLM for predicting text/image tokens with an auxiliary diffusion decoder for pixel generation. Employs unified training over both understanding and generation data.

Result: Achieves state-of-the-art results among unified models, competitive with specialist models, particularly on text-rich evaluation. Shows minimal task conflicts and consistent gains from scaling model size.

Conclusion: The hybrid tokenizer design is validated as effective, enabling scalable joint learning of both visual understanding and generation capabilities with reduced performance trade-offs.

Abstract: Unified multimodal Large Language Models (LLMs) that can both understand and generate visual content hold immense potential. However, existing open-source models often suffer from a performance trade-off between these capabilities. We present Manzano, a simple and scalable unified framework that substantially reduces this tension by coupling a hybrid image tokenizer with a well-curated training recipe. A single shared vision encoder feeds two lightweight adapters that produce continuous embeddings for image-to-text understanding and discrete tokens for text-to-image generation within a common semantic space. A unified autoregressive LLM predicts high-level semantics in the form of text and image tokens, with an auxiliary diffusion decoder subsequently translating the image tokens into pixels. The architecture, together with a unified training recipe over understanding and generation data, enables scalable joint learning of both capabilities. Manzano achieves state-of-the-art results among unified models, and is competitive with specialist models, particularly on text-rich evaluation. Our studies show minimal task conflicts and consistent gains from scaling model size, validating our design choice of a hybrid tokenizer.

Method: Created MCOD dataset with three key features: comprehensive challenge attributes (small objects, extreme lighting), diverse real-world scenarios, and high-quality pixel-level annotations. Benchmarked 11 representative COD methods on the dataset.

Result: Observed consistent performance drop in existing methods due to increased task difficulty, but multispectral modality integration substantially alleviated this degradation, highlighting the value of spectral information.

Conclusion: MCOD provides a strong foundation for future multispectral COD research and demonstrates that spectral information enhances detection robustness in challenging conditions.

Abstract: Camouflaged Object Detection (COD) aims to identify objects that blend seamlessly into natural scenes. Although RGB-based methods have advanced, their performance remains limited under challenging conditions. Multispectral imagery, providing rich spectral information, offers a promising alternative for enhanced foreground-background discrimination. However, existing COD benchmark datasets are exclusively RGB-based, lacking essential support for multispectral approaches, which has impeded progress in this area. To address this gap, we introduce MCOD, the first challenging benchmark dataset specifically designed for multispectral camouflaged object detection. MCOD features three key advantages: (i) Comprehensive challenge attributes: It captures real-world difficulties such as small object sizes and extreme lighting conditions commonly encountered in COD tasks. (ii) Diverse real-world scenarios: The dataset spans a wide range of natural environments to better reflect practical applications. (iii) High-quality pixel-level annotations: Each image is manually annotated with precise object masks and corresponding challenge attribute labels. We benchmark eleven representative COD methods on MCOD, observing a consistent performance drop due to increased task difficulty. Notably, integrating multispectral modalities substantially alleviates this degradation, highlighting the value of spectral information in enhancing detection robustness. We anticipate MCOD will provide a strong foundation for future research in multispectral camouflaged object detection. The dataset is publicly accessible at https://github.com/yl2900260-bit/MCOD.

Method: The challenge uses a weakly-labeled multi-label classification task where participants predict all plant species present in high-resolution plot images using single-label training data, with pre-trained vision transformer models provided.

Result: The paper provides detailed descriptions of the dataset, evaluation methodology, participant methods/models, and achieved results from the PlantCLEF 2024 challenge.

Conclusion: The PlantCLEF 2024 challenge establishes a comprehensive benchmark for advancing AI applications in ecological research, particularly for automated plant species identification in plot images.

Abstract: Plot images are essential for ecological studies, enabling standardized sampling, biodiversity assessment, long-term monitoring and remote, large-scale surveys. Plot images are typically fifty centimetres or one square meter in size, and botanists meticulously identify all the species found there. The integration of AI could significantly improve the efficiency of specialists, helping them to extend the scope and coverage of ecological studies. To evaluate advances in this regard, the PlantCLEF 2024 challenge leverages a new test set of thousands of multi-label images annotated by experts and covering over 800 species. In addition, it provides a large training set of 1.7 million individual plant images as well as state-of-the-art vision transformer models pre-trained on this data. The task is evaluated as a (weakly-labeled) multi-label classification task where the aim is to predict all the plant species present on a high-resolution plot image (using the single-label training data). In this paper, we provide an detailed description of the data, the evaluation methodology, the methods and models employed by the participants and the results achieved.

Method: The framework uses: 1) dual-path contrastive learning to create geometric-semantic fused embeddings, 2) coarse-to-fine registration with superpoint-superpixel correspondences and geometry-constrained refinement, and 3) dynamic training with gradient normalization to balance feature alignment, correspondence refinement, and pose estimation losses.

Result: CrossI2P outperforms state-of-the-art methods by 23.7% on KITTI Odometry benchmark and by 37.9% on nuScenes, showing significant improvements in both accuracy and robustness for autonomous driving applications.

Conclusion: The proposed self-supervised framework successfully bridges 2D and 3D modalities through unified cross-modal learning and two-stage registration, demonstrating superior performance and robustness compared to existing methods.

Abstract: Bridging 2D and 3D sensor modalities is critical for robust perception in autonomous systems. However, image-to-point cloud (I2P) registration remains challenging due to the semantic-geometric gap between texture-rich but depth-ambiguous images and sparse yet metrically precise point clouds, as well as the tendency of existing methods to converge to local optima. To overcome these limitations, we introduce CrossI2P, a self-supervised framework that unifies cross-modal learning and two-stage registration in a single end-to-end pipeline. First, we learn a geometric-semantic fused embedding space via dual-path contrastive learning, enabling annotation-free, bidirectional alignment of 2D textures and 3D structures. Second, we adopt a coarse-to-fine registration paradigm: a global stage establishes superpoint-superpixel correspondences through joint intra-modal context and cross-modal interaction modeling, followed by a geometry-constrained point-level refinement for precise registration. Third, we employ a dynamic training mechanism with gradient normalization to balance losses for feature alignment, correspondence refinement, and pose estimation. Extensive experiments demonstrate that CrossI2P outperforms state-of-the-art methods by 23.7% on the KITTI Odometry benchmark and by 37.9% on nuScenes, significantly improving both accuracy and robustness.

Method: VLM3D integrates VLMs as differentiable semantic and spatial priors into the SDS pipeline, leveraging Qwen2.5-VL model’s rich language-grounded supervision and spatial understanding capabilities.

Result: VLM3D significantly outperforms prior SDS-based methods on GPTeval3D benchmark, showing improvements in semantic fidelity, geometric coherence, and spatial correctness across diverse objects and complex scenes.

Conclusion: The integration of VLMs into SDS pipelines provides a powerful solution for text-to-3D generation, enabling fine-grained prompt alignment and enhanced 3D consistency through better semantic and spatial priors.

Abstract: Score Distillation Sampling (SDS) enables high-quality text-to-3D generation by supervising 3D models through the denoising of multi-view 2D renderings, using a pretrained text-to-image diffusion model to align with the input prompt and ensure 3D consistency. However, existing SDS-based methods face two fundamental limitations: (1) their reliance on CLIP-style text encoders leads to coarse semantic alignment and struggles with fine-grained prompts; and (2) 2D diffusion priors lack explicit 3D spatial constraints, resulting in geometric inconsistencies and inaccurate object relationships in multi-object scenes. To address these challenges, we propose VLM3D, a novel text-to-3D generation framework that integrates large vision-language models (VLMs) into the SDS pipeline as differentiable semantic and spatial priors. Unlike standard text-to-image diffusion priors, VLMs leverage rich language-grounded supervision that enables fine-grained prompt alignment. Moreover, their inherent vision language modeling provides strong spatial understanding, which significantly enhances 3D consistency for single-object generation and improves relational reasoning in multi-object scenes. We instantiate VLM3D based on the open-source Qwen2.5-VL model and evaluate it on the GPTeval3D benchmark. Experiments across diverse objects and complex scenes show that VLM3D significantly outperforms prior SDS-based methods in semantic fidelity, geometric coherence, and spatial correctness.

Method: Proposes RACap which mines structured relation semantics from retrieval captions and identifies heterogeneous objects from images, effectively retrieving structured relation features containing heterogeneous visual information to enhance semantic consistency and relational expressiveness.

Result: RACap achieves superior performance compared to previous lightweight captioning models with only 10.8M trainable parameters.

Conclusion: The proposed RACap model successfully addresses relation modeling challenges in image captioning by incorporating fine-grained relationship modeling and explicit object relationship identification, demonstrating effective performance with minimal parameters.

Abstract: Recent retrieval-augmented image captioning methods incorporate external knowledge to compensate for the limitations in comprehending complex scenes. However, current approaches face challenges in relation modeling: (1) the representation of semantic prompts is too coarse-grained to capture fine-grained relationships; (2) these methods lack explicit modeling of image objects and their semantic relationships. To address these limitations, we propose RACap, a relation-aware retrieval-augmented model for image captioning, which not only mines structured relation semantics from retrieval captions, but also identifies heterogeneous objects from the image. RACap effectively retrieves structured relation features that contain heterogeneous visual information to enhance the semantic consistency and relational expressiveness. Experimental results show that RACap, with only 10.8M trainable parameters, achieves superior performance compared to previous lightweight captioning models.

Method: Replace Cutie’s encoder with SAM2’s pretrained ViT encoder, introduce motion prediction module for temporal stability, and use ensemble strategy combining Cutie, SAM2, and the proposed variant

Result: Achieved 3rd place in the MOSEv2 track of the 7th LSVOS Challenge

Conclusion: The framework demonstrates effectiveness of enriched feature representation and motion prediction for robust video object segmentation

Abstract: Video object segmentation (VOS) is a challenging task with wide applications such as video editing and autonomous driving. While Cutie provides strong query-based segmentation and SAM2 offers enriched representations via a pretrained ViT encoder, each has limitations in feature capacity and temporal modeling. In this report, we propose a framework that integrates their complementary strengths by replacing the encoder of Cutie with the ViT encoder of SAM2 and introducing a motion prediction module for temporal stability. We further adopt an ensemble strategy combining Cutie, SAM2, and our variant, achieving 3rd place in the MOSEv2 track of the 7th LSVOS Challenge. We refer to our final model as SCOPE (SAM2-CUTIE Object Prediction Ensemble). This demonstrates the effectiveness of enriched feature representation and motion prediction for robust video object segmentation. The code is available at https://github.com/2025-LSVOS-3rd-place/MOSEv2_3rd_place.

Method: Proposes FoBa method with foreground-background co-guidance to leverage both regions of interest and contextual information. Includes Gated Interaction Fusion (GIF) module and consistency loss for bi-temporal interaction and spatial consistency.

Result: FoBa achieves competitive results on three datasets (SECOND, JL1, and LevirSCD) with improvements of 1.48%, 3.61%, and 2.81% in SeK metric respectively compared to SOTA methods.

Conclusion: The proposed LevirSCD dataset and FoBa method effectively address data and methodological limitations in remote sensing SCD, demonstrating superior performance through collaborative foreground-background guidance and improved change information utilization.

Abstract: Despite the remarkable progress achieved in remote sensing semantic change detection (SCD), two major challenges remain. At the data level, existing SCD datasets suffer from limited change categories, insufficient change types, and a lack of fine-grained class definitions, making them inadequate to fully support practical applications. At the methodological level, most current approaches underutilize change information, typically treating it as a post-processing step to enhance spatial consistency, which constrains further improvements in model performance. To address these issues, we construct a new benchmark for remote sensing SCD, LevirSCD. Focused on the Beijing area, the dataset covers 16 change categories and 210 specific change types, with more fine-grained class definitions (e.g., roads are divided into unpaved and paved roads). Furthermore, we propose a foreground-background co-guided SCD (FoBa) method, which leverages foregrounds that focus on regions of interest and backgrounds enriched with contextual information to guide the model collaboratively, thereby alleviating semantic ambiguity while enhancing its ability to detect subtle changes. Considering the requirements of bi-temporal interaction and spatial consistency in SCD, we introduce a Gated Interaction Fusion (GIF) module along with a simple consistency loss to further enhance the model’s detection performance. Extensive experiments on three datasets (SECOND, JL1, and the proposed LevirSCD) demonstrate that FoBa achieves competitive results compared to current SOTA methods, with improvements of 1.48%, 3.61%, and 2.81% in the SeK metric, respectively. Our code and dataset are available at https://github.com/zmoka-zht/FoBa.

Method: The authors propose MS-UDG, which integrates: (1) an InfoNCE-based objective for sufficiency, (2) a novel semantic-variation disentanglement loss, and (3) a reconstruction-based mechanism for capturing adequate variation to promote minimality. The approach is theoretically grounded in information theory.

Result: MS-UDG achieves state-of-the-art performance on popular unsupervised domain-generalization benchmarks, consistently outperforming existing SSL and UDG methods without requiring category or domain labels during representation learning.

Conclusion: The proposed MS-UDG framework effectively addresses unsupervised domain generalization by learning minimal sufficient semantic representations, demonstrating superior generalization ability without the need for domain labels that are often unavailable in practical applications.

Abstract: The generalization ability of deep learning has been extensively studied in supervised settings, yet it remains less explored in unsupervised scenarios. Recently, the Unsupervised Domain Generalization (UDG) task has been proposed to enhance the generalization of models trained with prevalent unsupervised learning techniques, such as Self-Supervised Learning (SSL). UDG confronts the challenge of distinguishing semantics from variations without category labels. Although some recent methods have employed domain labels to tackle this issue, such domain labels are often unavailable in real-world contexts. In this paper, we address these limitations by formalizing UDG as the task of learning a Minimal Sufficient Semantic Representation: a representation that (i) preserves all semantic information shared across augmented views (sufficiency), and (ii) maximally removes information irrelevant to semantics (minimality). We theoretically ground these objectives from the perspective of information theory, demonstrating that optimizing representations to achieve sufficiency and minimality directly reduces out-of-distribution risk. Practically, we implement this optimization through Minimal-Sufficient UDG (MS-UDG), a learnable model by integrating (a) an InfoNCE-based objective to achieve sufficiency; (b) two complementary components to promote minimality: a novel semantic-variation disentanglement loss and a reconstruction-based mechanism for capturing adequate variation. Empirically, MS-UDG sets a new state-of-the-art on popular unsupervised domain-generalization benchmarks, consistently outperforming existing SSL and UDG methods, without category or domain labels during representation learning.

Method: Integrates three modules: terrain-aware adapter for elevation priors, temporal prompt generator for land-cover changes, and multi-scale fusion for fine-grained delineation.

Result: Substantial performance gains on LoveDA, iSAID, and WHU-CD benchmarks, outperforming zero-shot SAM and task-specific models with minimal overhead.

Conclusion: Domain-adaptive augmentation enhances foundation models for geospatial segmentation, offering a scalable solution.

Abstract: Segment Anything Model (SAM) has demonstrated impressive zero-shot segmentation capabilities across natural image domains, but it struggles to generalize to the unique challenges of remote sensing data, such as complex terrain, multi-scale objects, and temporal dynamics. In this paper, we introduce TASAM, a terrain and temporally-aware extension of SAM designed specifically for high-resolution remote sensing image segmentation. TASAM integrates three lightweight yet effective modules: a terrain-aware adapter that injects elevation priors, a temporal prompt generator that captures land-cover changes over time, and a multi-scale fusion strategy that enhances fine-grained object delineation. Without retraining the SAM backbone, our approach achieves substantial performance gains across three remote sensing benchmarks-LoveDA, iSAID, and WHU-CD-outperforming both zero-shot SAM and task-specific models with minimal computational overhead. Our results highlight the value of domain-adaptive augmentation for foundation models and offer a scalable path toward more robust geospatial segmentation.

Method: Proposes a teacher-student framework where the teacher is the task model and the student is an auxiliary model that learns from the teacher. Both models are trained simultaneously in each AL cycle, using distance between their outputs to measure uncertainty for unlabeled data.

Result: The method is validated on classical computer vision tasks and cryo-ET challenges, demonstrating efficacy and efficiency without requiring special training techniques.

Conclusion: The proposed knowledge transfer approach provides an effective and efficient uncertainty estimation method for Active Learning that is task-agnostic and suitable for various domains.

Abstract: Uncertainty estimation is at the core of Active Learning (AL). Most existing methods resort to complex auxiliary models and advanced training fashions to estimate uncertainty for unlabeled data. These models need special design and hence are difficult to train especially for domain tasks, such as Cryo-Electron Tomography (cryo-ET) classification in computational biology. To address this challenge, we propose a novel method using knowledge transfer to boost uncertainty estimation in AL. Specifically, we exploit the teacher-student mode where the teacher is the task model in AL and the student is an auxiliary model that learns from the teacher. We train the two models simultaneously in each AL cycle and adopt a certain distance between the model outputs to measure uncertainty for unlabeled data. The student model is task-agnostic and does not rely on special training fashions (e.g. adversarial), making our method suitable for various tasks. More importantly, we demonstrate that data uncertainty is not tied to concrete value of task loss but closely related to the upper-bound of task loss. We conduct extensive experiments to validate the proposed method on classical computer vision tasks and cryo-ET challenges. The results demonstrate its efficacy and efficiency.

Method: LC-SLab supports two object-based approaches: input-level aggregation via graph neural networks and output-level aggregation by postprocessing semantic segmentation results. It incorporates features from large pre-trained networks to improve small dataset performance, evaluated on Sentinel-2 composites with sparse LUCAS labels.

Result: Object-based methods match or exceed pixel-wise model accuracy while producing more coherent maps. Input-level aggregation is more robust on smaller datasets, while output-level aggregation performs best with more data. Several LC-SLab configurations outperform existing land cover products.

Conclusion: LC-SLab demonstrates that object-based deep learning methods effectively address fragmentation issues in land cover mapping under sparse supervision, offering practical utility for large-scale applications with a clear tradeoff between accuracy and coherence.

Abstract: Large-scale land cover maps generated using deep learning play a critical role across a wide range of Earth science applications. Open in-situ datasets from principled land cover surveys offer a scalable alternative to manual annotation for training such models. However, their sparse spatial coverage often leads to fragmented and noisy predictions when used with existing deep learning-based land cover mapping approaches. A promising direction to address this issue is object-based classification, which assigns labels to semantically coherent image regions rather than individual pixels, thereby imposing a minimum mapping unit. Despite this potential, object-based methods remain underexplored in deep learning-based land cover mapping pipelines, especially in the context of medium-resolution imagery and sparse supervision. To address this gap, we propose LC-SLab, the first deep learning framework for systematically exploring object-based deep learning methods for large-scale land cover classification under sparse supervision. LC-SLab supports both input-level aggregation via graph neural networks, and output-level aggregation by postprocessing results from established semantic segmentation models. Additionally, we incorporate features from a large pre-trained network to improve performance on small datasets. We evaluate the framework on annual Sentinel-2 composites with sparse LUCAS labels, focusing on the tradeoff between accuracy and fragmentation, as well as sensitivity to dataset size. Our results show that object-based methods can match or exceed the accuracy of common pixel-wise models while producing substantially more coherent maps. Input-level aggregation proves more robust on smaller datasets, whereas output-level aggregation performs best with more data. Several configurations of LC-SLab also outperform existing land cover products, highlighting the framework’s practical utility.

Method: Uses SegResNet-based encoder with prompt encoder and mask decoder in U-Net architecture, incorporating latent cross-attention, relative positional encoding, normalized attention, and Muon optimizer. Trained from scratch on <5,000 volumes from multiple modalities on a single 32GB GPU.

Result: Achieved DSC AUC of 2.404, NSD AUC of 2.266, final DSC of 0.627, and final NSD of 0.597 on hidden test set, outperforming VISTA3D and SAM-Med3D, matching SegVol. Ranked 5th overall and best among non-pretrained approaches in the challenge.

Conclusion: ENSAM demonstrates that effective 3D medical image segmentation can be achieved with limited resources, with relative positional encodings and Muon optimizer being key contributors to its performance and convergence speed.

Abstract: We present ENSAM (Equivariant, Normalized, Segment Anything Model), a lightweight and promptable model for universal 3D medical image segmentation. ENSAM combines a SegResNet-based encoder with a prompt encoder and mask decoder in a U-Net-style architecture, using latent cross-attention, relative positional encoding, normalized attention, and the Muon optimizer for training. ENSAM is designed to achieve good performance under limited data and computational budgets, and is trained from scratch on under 5,000 volumes from multiple modalities (CT, MRI, PET, ultrasound, microscopy) on a single 32 GB GPU in 6 hours. As part of the CVPR 2025 Foundation Models for Interactive 3D Biomedical Image Segmentation Challenge, ENSAM was evaluated on hidden test set with multimodal 3D medical images, obtaining a DSC AUC of 2.404, NSD AUC of 2.266, final DSC of 0.627, and final NSD of 0.597, outperforming two previously published baseline models (VISTA3D, SAM-Med3D) and matching the third (SegVol), surpassing its performance in final DSC but trailing behind in the other three metrics. In the coreset track of the challenge, ENSAM ranks 5th of 10 overall and best among the approaches not utilizing pretrained weights. Ablation studies confirm that our use of relative positional encodings and the Muon optimizer each substantially speed up convergence and improve segmentation quality.

Method: The framework adapts SAM2 to 3D segmentation using projection/back-projection techniques with architectural modifications: (1) novel module for horizontal spatial dependencies in LiDAR range images, (2) customized configuration for spherical projections, (3) adapted mechanism for range-view spatial patterns and discontinuities.

Result: The approach achieves competitive performance on SemanticKITTI while benefiting from the speed, scalability, and deployment simplicity of 2D-centric pipelines.

Conclusion: Range-view segmentation methods using Visual Foundation Models lead to promising results, highlighting VFMs as viable general-purpose backbones for 3D perception and opening a path toward unified foundation-model-driven LiDAR segmentation.

Abstract: Point cloud segmentation is central to autonomous driving and 3D scene understanding. While voxel- and point-based methods dominate recent research due to their compatibility with deep architectures and ability to capture fine-grained geometry, they often incur high computational cost, irregular memory access, and limited real-time efficiency. In contrast, range-view methods, though relatively underexplored - can leverage mature 2D semantic segmentation techniques for fast and accurate predictions. Motivated by the rapid progress in Visual Foundation Models (VFMs) for captioning, zero-shot recognition, and multimodal tasks, we investigate whether SAM2, the current state-of-the-art VFM for segmentation tasks, can serve as a strong backbone for LiDAR point cloud segmentation in the range view. We present , to our knowledge, the first range-view framework that adapts SAM2 to 3D segmentation, coupling efficient 2D feature extraction with standard projection/back-projection to operate on point clouds. To optimize SAM2 for range-view representations, we implement several architectural modifications to the encoder: (1) a novel module that emphasizes horizontal spatial dependencies inherent in LiDAR range images, (2) a customized configuration of tailored to the geometric properties of spherical projections, and (3) an adapted mechanism in the encoder backbone specifically designed to capture the unique spatial patterns and discontinuities present in range-view pseudo-images. Our approach achieves competitive performance on SemanticKITTI while benefiting from the speed, scalability, and deployment simplicity of 2D-centric pipelines. This work highlights the viability of VFMs as general-purpose backbones for 3D perception and opens a path toward unified, foundation-model-driven LiDAR segmentation. Results lets us conclude that range-view segmentation methods using VFMs leads to promising results.

Method: Proposes GREAT framework with three attention modules: Spatial Attention (SA) for global context in spatial dimension, Matching Attention (MA) for global context along epipolar lines, and Volume Attention (VA) that works with SA/MA to build robust cost-volume excited by global context and geometric details.

Result: GREAT-IGEV ranks first on Scene Flow test set, KITTI 2015, and ETH3D leaderboards, and second on Middlebury benchmark among all published methods. The framework demonstrates superior performance in challenging ill-posed regions.

Conclusion: The GREAT framework effectively enables iterative stereo matching methods to incorporate global context, showing universality and superior performance across multiple representative methods and benchmarks.

Abstract: Stereo matching achieves significant progress with iterative algorithms like RAFT-Stereo and IGEV-Stereo. However, these methods struggle in ill-posed regions with occlusions, textureless, or repetitive patterns, due to a lack of global context and geometric information for effective iterative refinement. To enable the existing iterative approaches to incorporate global context, we propose the Global Regulation and Excitation via Attention Tuning (GREAT) framework which encompasses three attention modules. Specifically, Spatial Attention (SA) captures the global context within the spatial dimension, Matching Attention (MA) extracts global context along epipolar lines, and Volume Attention (VA) works in conjunction with SA and MA to construct a more robust cost-volume excited by global context and geometric details. To verify the universality and effectiveness of this framework, we integrate it into several representative iterative stereo-matching methods and validate it through extensive experiments, collectively denoted as GREAT-Stereo. This framework demonstrates superior performance in challenging ill-posed regions. Applied to IGEV-Stereo, among all published methods, our GREAT-IGEV ranks first on the Scene Flow test set, KITTI 2015, and ETH3D leaderboards, and achieves second on the Middlebury benchmark. Code is available at https://github.com/JarvisLee0423/GREAT-Stereo.

Method: Model the feedback process as a differential equation solved through a recurrent neural network, stabilized via exponential decay to ensure convergence. Evaluated on object recognition and segmentation tasks under noise and limited data conditions.

Result: DFMs consistently outperform feedforward counterparts in both object recognition and segmentation, particularly in low data or high noise regimes. They also translate well to medical imaging and show robustness against various noise corruptions.

Conclusion: Feedback mechanisms are crucial for achieving stable, robust, and generalizable learning in neural networks, with DFMs demonstrating superior performance in challenging conditions.

Abstract: Deep Feedback Models (DFMs) are a new class of stateful neural networks that combine bottom up input with high level representations over time. This feedback mechanism introduces dynamics into otherwise static architectures, enabling DFMs to iteratively refine their internal state and mimic aspects of biological decision making. We model this process as a differential equation solved through a recurrent neural network, stabilized via exponential decay to ensure convergence. To evaluate their effectiveness, we measure DFMs under two key conditions: robustness to noise and generalization with limited data. In both object recognition and segmentation tasks, DFMs consistently outperform their feedforward counterparts, particularly in low data or high noise regimes. In addition, DFMs translate to medical imaging settings, while being robust against various types of noise corruption. These findings highlight the importance of feedback in achieving stable, robust, and generalizable learning. Code is available at https://github.com/DCalhas/deep_feedback_models.

Method: Leverages pre-trained 2D foundation models without 3D-specific training. Lifts 2D detections and optimizes 3D proposals for featuremetric consistency across views, avoiding computationally expensive 3D feature fusion.

Result: Establishes a powerful baseline that performs competitively with state-of-the-art methods in dense scenarios and significantly outperforms them in sparse-view settings.

Conclusion: Simple pipeline effectively utilizes 2D training data advantages over 3D, demonstrating strong performance for open-vocabulary 3D object detection in sparse-view conditions.

Abstract: The ability to interpret and comprehend a 3D scene is essential for many vision and robotics systems. In numerous applications, this involves 3D object detection, i.e.~identifying the location and dimensions of objects belonging to a specific category, typically represented as bounding boxes. This has traditionally been solved by training to detect a fixed set of categories, which limits its use. In this work, we investigate open-vocabulary 3D object detection in the challenging yet practical sparse-view setting, where only a limited number of posed RGB images are available as input. Our approach is training-free, relying on pre-trained, off-the-shelf 2D foundation models instead of employing computationally expensive 3D feature fusion or requiring 3D-specific learning. By lifting 2D detections and directly optimizing 3D proposals for featuremetric consistency across views, we fully leverage the extensive training data available in 2D compared to 3D. Through standard benchmarks, we demonstrate that this simple pipeline establishes a powerful baseline, performing competitively with state-of-the-art techniques in densely sampled scenarios while significantly outperforming them in the sparse-view setting.

Method: The study investigates feature attribution methods (Saliency Maps, Integrated Gradients, Attention Rollout) on MDE models (METER and PixelFormer), evaluates explanations through selective pixel perturbation, and introduces Attribution Fidelity metric to assess explanation reliability.

Result: Saliency Maps perform well for lightweight MDE models, while Integrated Gradients work better for deep models. Attribution Fidelity effectively identifies when explainability methods fail to produce reliable visual maps, even when conventional metrics show satisfactory results.

Conclusion: The research demonstrates that different explainability methods suit different MDE model complexities, and Attribution Fidelity provides a more reliable way to evaluate explanation quality than existing metrics.

Abstract: Explainable artificial intelligence is increasingly employed to understand the decision-making process of deep learning models and create trustworthiness in their adoption. However, the explainability of Monocular Depth Estimation (MDE) remains largely unexplored despite its wide deployment in real-world applications. In this work, we study how to analyze MDE networks to map the input image to the predicted depth map. More in detail, we investigate well-established feature attribution methods, Saliency Maps, Integrated Gradients, and Attention Rollout on different computationally complex models for MDE: METER, a lightweight network, and PixelFormer, a deep network. We assess the quality of the generated visual explanations by selectively perturbing the most relevant and irrelevant pixels, as identified by the explainability methods, and analyzing the impact of these perturbations on the model’s output. Moreover, since existing evaluation metrics can have some limitations in measuring the validity of visual explanations for MDE, we additionally introduce the Attribution Fidelity. This metric evaluates the reliability of the feature attribution by assessing their consistency with the predicted depth map. Experimental results demonstrate that Saliency Maps and Integrated Gradients have good performance in highlighting the most important input features for MDE lightweight and deep models, respectively. Furthermore, we show that Attribution Fidelity effectively identifies whether an explainability method fails to produce reliable visual maps, even in scenarios where conventional metrics might suggest satisfactory results.

Method: Proposes a new backbone that maps radar pillar features into embedded dimensions using self-attention to model radar point dependencies, replaces FPN-based convolutional layers with simplified convolutional layers in PointPillars-based architectures to reduce inference time, and fuses features in bird’s-eye-view (BEV) representation.

Result: Achieves state-of-the-art performance with 58.2 NDS metric using ResNet-50, while setting new benchmark for inference time on nuScenes dataset in the same category.

Conclusion: The proposed camera-radar fusion approach effectively leverages radar advantages and achieves superior performance with efficient inference, demonstrating the potential of radar-camera systems as cost-effective alternatives to lidar-based solutions.

Abstract: Camera-radar fusion offers a robust and low-cost alternative to Camera-lidar fusion for the 3D object detection task in real-time under adverse weather and lighting conditions. However, currently, in the literature, it is possible to find few works focusing on this modality and, most importantly, developing new architectures to explore the advantages of the radar point cloud, such as accurate distance estimation and speed information. Therefore, this work presents a novel and efficient 3D object detection algorithm using cameras and radars in the bird’s-eye-view (BEV). Our algorithm exploits the advantages of radar before fusing the features into a detection head. A new backbone is introduced, which maps the radar pillar features into an embedded dimension. A self-attention mechanism allows the backbone to model the dependencies between the radar points. We are using a simplified convolutional layer to replace the FPN-based convolutional layers used in the PointPillars-based architectures with the main goal of reducing inference time. Our results show that with this modification, our approach achieves the new state-of-the-art in the 3D object detection problem, reaching 58.2 of the NDS metric for the use of ResNet-50, while also setting a new benchmark for inference time on the nuScenes dataset for the same category.

Method: Models per-pixel depth as a mixture distribution to capture multiple plausible depths, shifting uncertainty from direct regression to mixture weights. Integrates into existing pipelines via variance-aware loss functions and uncertainty propagation.

Result: Achieves up to 35% higher boundary sharpness and improves point cloud quality compared to state-of-the-art baselines on KITTI and VKITTIv2 datasets.

Conclusion: The method successfully produces crisp depth discontinuities using only self-supervision, demonstrating significant improvements in boundary sharpness and point cloud quality over existing approaches.

Abstract: Accurate monocular depth estimation is crucial for 3D scene understanding, but existing methods often blur depth at object boundaries, introducing spurious intermediate 3D points. While achieving sharp edges usually requires very fine-grained supervision, our method produces crisp depth discontinuities using only self-supervision. Specifically, we model per-pixel depth as a mixture distribution, capturing multiple plausible depths and shifting uncertainty from direct regression to the mixture weights. This formulation integrates seamlessly into existing pipelines via variance-aware loss functions and uncertainty propagation. Extensive evaluations on KITTI and VKITTIv2 show that our method achieves up to 35% higher boundary sharpness and improves point cloud quality compared to state-of-the-art baselines.

Method: MTMS-YieldNet uses contrastive learning for pre-training to capture spatial-spectral patterns and spatio-temporal dependencies from remote sensing data, integrating multi-spectral data with spatio-temporal information for effective yield prediction.

Result: Achieves MAPE scores of 0.336 on Sentinel-1, 0.353 on Landsat-8, and 0.331 on Sentinel-2, outperforming seven existing state-of-the-art methods across diverse climatic and seasonal conditions.

Conclusion: MTMS-YieldNet significantly improves yield prediction accuracy and provides valuable insights for farmers’ decision-making, potentially enhancing crop yields and contributing to agricultural sustainability.

Abstract: Precise yield prediction is essential for agricultural sustainability and food security. However, climate change complicates accurate yield prediction by affecting major factors such as weather conditions, soil fertility, and farm management systems. Advances in technology have played an essential role in overcoming these challenges by leveraging satellite monitoring and data analysis for precise yield estimation. Current methods rely on spatio-temporal data for predicting crop yield, but they often struggle with multi-spectral data, which is crucial for evaluating crop health and growth patterns. To resolve this challenge, we propose a novel Multi-Temporal Multi-Spectral Yield Prediction Network, MTMS-YieldNet, that integrates spectral data with spatio-temporal information to effectively capture the correlations and dependencies between them. While existing methods that rely on pre-trained models trained on general visual data, MTMS-YieldNet utilizes contrastive learning for feature discrimination during pre-training, focusing on capturing spatial-spectral patterns and spatio-temporal dependencies from remote sensing data. Both quantitative and qualitative assessments highlight the excellence of the proposed MTMS-YieldNet over seven existing state-of-the-art methods. MTMS-YieldNet achieves MAPE scores of 0.336 on Sentinel-1, 0.353 on Landsat-8, and an outstanding 0.331 on Sentinel-2, demonstrating effective yield prediction performance across diverse climatic and seasonal conditions. The outstanding performance of MTMS-YieldNet improves yield predictions and provides valuable insights that can assist farmers in making better decisions, potentially improving crop yields.

Method: Asymmetric fusion strategy that begins with a primary modality and integrates secondary modalities by disentangling shared and modality-specific information.

Result: Outperforms existing methods on a dataset of 239 patients with echocardiographic time series and tabular records, achieving an AUC over 90%.

Conclusion: The improvement marks a crucial benchmark for clinical use, demonstrating the effectiveness of the proposed asymmetric fusion approach.

Abstract: Multimodal data fusion is a key approach for enhancing diagnosis in medical applications. We propose an asymmetric fusion strategy starting from a primary modality and integrating secondary modalities by disentangling shared and modality-specific information. Validated on a dataset of 239 patients with echocardiographic time series and tabular records, our model outperforms existing methods, achieving an AUC over 90%. This improvement marks a crucial benchmark for clinical use.

Method: Uses lightweight LLM agent for category disambiguation and visual-modal expansion to generate multiple semantic prototypes, with LogSumExp aggregation for adaptive alignment between vision model and relevant prototypes.

Result: Extensive experiments show MuproCL consistently enhances performance and robustness across various continual learning baselines.

Conclusion: Multiple context-aware prototypes provide a more effective path for language-guided continual learning compared to single-target approaches.

Abstract: Language-guided supervision, which utilizes a frozen semantic target from a Pretrained Language Model (PLM), has emerged as a promising paradigm for visual Continual Learning (CL). However, relying on a single target introduces two critical limitations: 1) semantic ambiguity, where a polysemous category name results in conflicting visual representations, and 2) intra-class visual diversity, where a single prototype fails to capture the rich variety of visual appearances within a class. To this end, we propose MuproCL, a novel framework that replaces the single target with multiple, context-aware prototypes. Specifically, we employ a lightweight LLM agent to perform category disambiguation and visual-modal expansion to generate a robust set of semantic prototypes. A LogSumExp aggregation mechanism allows the vision model to adaptively align with the most relevant prototype for a given image. Extensive experiments across various CL baselines demonstrate that MuproCL consistently enhances performance and robustness, establishing a more effective path for language-guided continual learning.

Method: Uses knowledge distillation from VFMs to build lightweight student model; extracts high-level semantic features; injects modality category information; employs V2I-GAN for visible-to-infrared translation for data augmentation.

Result: Experiments show DistillMatch outperforms existing algorithms on public datasets.

Conclusion: The proposed method effectively addresses multimodal image matching challenges by leveraging VFM knowledge distillation and modality-specific information injection, demonstrating superior performance over existing approaches.

Abstract: Multimodal image matching seeks pixel-level correspondences between images of different modalities, crucial for cross-modal perception, fusion and analysis. However, the significant appearance differences between modalities make this task challenging. Due to the scarcity of high-quality annotated datasets, existing deep learning methods that extract modality-common features for matching perform poorly and lack adaptability to diverse scenarios. Vision Foundation Model (VFM), trained on large-scale data, yields generalizable and robust feature representations adapted to data and tasks of various modalities, including multimodal matching. Thus, we propose DistillMatch, a multimodal image matching method using knowledge distillation from VFM. DistillMatch employs knowledge distillation to build a lightweight student model that extracts high-level semantic features from VFM (including DINOv2 and DINOv3) to assist matching across modalities. To retain modality-specific information, it extracts and injects modality category information into the other modality’s features, which enhances the model’s understanding of cross-modal correlations. Furthermore, we design V2I-GAN to boost the model’s generalization by translating visible to pseudo-infrared images for data augmentation. Experiments show that DistillMatch outperforms existing algorithms on public datasets.

Method: Uses Maximum Semantic Richness Sampling to extract semantically rich keyframes for visual diversity, and simulates visual trajectories with encoded relative spatial positions for motion reconstruction. The method requires only a single forward pass and integrates seamlessly with existing MLLMs.

Result: Extensive experiments on VSI-BENCH and STI-BENCH show consistent performance boosts across diverse spatial reasoning tasks, with up to +3.5% improvement in MLLM performance.

Conclusion: SEE&TREK offers a promising path toward stronger spatial intelligence in MLLMs through training-free visual prompting, demonstrating effective spatial understanding enhancement under vision-only constraints.

Abstract: We introduce SEE&TREK, the first training-free prompting framework tailored to enhance the spatial understanding of Multimodal Large Language Models (MLLMS) under vision-only constraints. While prior efforts have incorporated modalities like depth or point clouds to improve spatial reasoning, purely visualspatial understanding remains underexplored. SEE&TREK addresses this gap by focusing on two core principles: increasing visual diversity and motion reconstruction. For visual diversity, we conduct Maximum Semantic Richness Sampling, which employs an off-the-shell perception model to extract semantically rich keyframes that capture scene structure. For motion reconstruction, we simulate visual trajectories and encode relative spatial positions into keyframes to preserve both spatial relations and temporal coherence. Our method is training&GPU-free, requiring only a single forward pass, and can be seamlessly integrated into existing MLLM’S. Extensive experiments on the VSI-B ENCH and STI-B ENCH show that S EE &T REK consistently boosts various MLLM S performance across diverse spatial reasoning tasks with the most +3.5% improvement, offering a promising path toward stronger spatial intelligence.

Method: The authors design CauMVC using causal modeling, treating partially aligned data as an intervention. They use a Variational Auto-Encoder with an encoder to estimate invariant features and a decoder for post-intervention inference, plus a contrastive regularizer for sample correlations.

Result: Empirical experiments on both fully and partially aligned data demonstrate that CauMVC achieves strong generalization and effectiveness in handling the generalized multi-view clustering problem.

Conclusion: This is the first work to address generalized multi-view clustering through causal learning, providing a novel solution that maintains performance even when data alignment is incomplete across views.

Abstract: Multi-view clustering (MVC) aims to explore the common clustering structure across multiple views. Many existing MVC methods heavily rely on the assumption of view consistency, where alignments for corresponding samples across different views are ordered in advance. However, real-world scenarios often present a challenge as only partial data is consistently aligned across different views, restricting the overall clustering performance. In this work, we consider the model performance decreasing phenomenon caused by data order shift (i.e., from fully to partially aligned) as a generalized multi-view clustering problem. To tackle this problem, we design a causal multi-view clustering network, termed CauMVC. We adopt a causal modeling approach to understand multi-view clustering procedure. To be specific, we formulate the partially aligned data as an intervention and multi-view clustering with partially aligned data as an post-intervention inference. However, obtaining invariant features directly can be challenging. Thus, we design a Variational Auto-Encoder for causal learning by incorporating an encoder from existing information to estimate the invariant features. Moreover, a decoder is designed to perform the post-intervention inference. Lastly, we design a contrastive regularizer to capture sample correlations. To the best of our knowledge, this paper is the first work to deal generalized multi-view clustering via causal learning. Empirical experiments on both fully and partially aligned data illustrate the strong generalization and effectiveness of CauMVC.

Method: GLip uses a dual-path feature extraction architecture integrating global and local features within a two-stage progressive learning framework. Stage 1 learns coarse alignment between visual features and speech units using audio-visual data. Stage 2 refines this with a Contextual Enhancement Module that dynamically integrates local features with global context across spatial and temporal dimensions.

Result: The framework consistently outperforms existing methods on LRS2 and LRS3 benchmarks and demonstrates effectiveness on a challenging Mandarin dataset, showing enhanced robustness against various visual challenges.

Conclusion: GLip’s progressive learning strategy that exploits discriminative local regions provides superior robustness for visual speech recognition in challenging real-world conditions.

Abstract: Visual speech recognition (VSR), also known as lip reading, is the task of recognizing speech from silent video. Despite significant advancements in VSR over recent decades, most existing methods pay limited attention to real-world visual challenges such as illumination variations, occlusions, blurring, and pose changes. To address these challenges, we propose GLip, a Global-Local Integrated Progressive framework designed for robust VSR. GLip is built upon two key insights: (i) learning an initial \textit{coarse} alignment between visual features across varying conditions and corresponding speech content facilitates the subsequent learning of \textit{precise} visual-to-speech mappings in challenging environments; (ii) under adverse conditions, certain local regions (e.g., non-occluded areas) often exhibit more discriminative cues for lip reading than global features. To this end, GLip introduces a dual-path feature extraction architecture that integrates both global and local features within a two-stage progressive learning framework. In the first stage, the model learns to align both global and local visual features with corresponding acoustic speech units using easily accessible audio-visual data, establishing a coarse yet semantically robust foundation. In the second stage, we introduce a Contextual Enhancement Module (CEM) to dynamically integrate local features with relevant global context across both spatial and temporal dimensions, refining the coarse representations into precise visual-speech mappings. Our framework uniquely exploits discriminative local regions through a progressive learning strategy, demonstrating enhanced robustness against various visual challenges and consistently outperforming existing methods on the LRS2 and LRS3 benchmarks. We further validate its effectiveness on a newly introduced challenging Mandarin dataset.

Method: Dictionary-Guided Graph Convolutional Network that simultaneously predicts both the location and number of control points for B-spline surface reconstruction, eliminating the need for point normals.

Result: The method outperforms several well-known and recent baselines both qualitatively and quantitatively on widely-used evaluation metrics.

Conclusion: The proposed approach successfully generates smooth surfaces from noisy point clouds without requiring normals, addressing key limitations of existing methods.

Abstract: Generating continuous surfaces from discrete point cloud data is a fundamental task in several 3D vision applications. Real-world point clouds are inherently noisy due to various technical and environmental factors. Existing data-driven surface reconstruction algorithms rely heavily on ground truth normals or compute approximate normals as an intermediate step. This dependency makes them extremely unreliable for noisy point cloud datasets, even if the availability of ground truth training data is ensured, which is not always the case. B-spline reconstruction techniques provide compact surface representations of point clouds and are especially known for their smoothening properties. However, the complexity of the surfaces approximated using B-splines is directly influenced by the number and location of the spline control points. Existing spline-based modeling methods predict the locations of a fixed number of control points for a given point cloud, which makes it very difficult to match the complexity of its underlying surface. In this work, we develop a Dictionary-Guided Graph Convolutional Network-based surface reconstruction strategy where we simultaneously predict both the location and the number of control points for noisy point cloud data to generate smooth surfaces without the use of any point normals. We compare our reconstruction method with several well-known as well as recent baselines by employing widely-used evaluation metrics, and demonstrate that our method outperforms all of them both qualitatively and quantitatively.

Method: Leverages the bisection method to exploit the unimodal characteristics of the between-class variance function, reducing the number of required evaluations while preserving the algorithm’s theoretical foundations.

Result: Experimental results on 48 test images show 91.63% reduction in variance computations, 97.21% reduction in iterations, with exact threshold matches in 66.67% of cases and 95.83% within 5 gray levels deviation.

Conclusion: The optimized algorithm provides deterministic performance guarantees suitable for real-time applications while maintaining the segmentation quality of the original Otsu method.

Abstract: The Otsu thresholding algorithm represents a fundamental technique in image segmentation, yet its computational efficiency is severely limited by exhaustive search requirements across all possible threshold values. This work presents an optimized implementation that leverages the bisection method to exploit the unimodal characteristics of the between-class variance function. Our approach reduces the computational complexity from O(L) to O(log L) evaluations while preserving segmentation accuracy. Experimental validation on 48 standard test images demonstrates a 91.63% reduction in variance computations and 97.21% reduction in algorithmic iterations compared to conventional exhaustive search. The bisection method achieves exact threshold matches in 66.67% of test cases, with 95.83% exhibiting deviations within 5 gray levels. The algorithm maintains universal convergence within theoretical logarithmic bounds while providing deterministic performance guarantees suitable for real-time applications. This optimization addresses critical computational bottlenecks in large-scale image processing systems without compromising the theoretical foundations or segmentation quality of the original Otsu method.

Method: The approach expands MLLM vocabulary with activity-level tokens and cluster-specific tokens. It processes video frames with these tokens and language instructions through MLLM, uses multi-label classification loss for semantic learning, and integrates embeddings with visual features via a Multimodal Dual-Alignment Fusion module.

Result: Both quantitative and qualitative experiments demonstrate superior performance in Group Activity Detection tasks compared to existing methods.

Conclusion: The proposed LIR-GAD framework effectively leverages MLLMs’ commonsense knowledge and language instructions to enhance GAD performance, addressing limitations of previous visual-only approaches.

Abstract: Group activity detection (GAD) aims to simultaneously identify group members and categorize their collective activities within video sequences. Existing deep learning-based methods develop specialized architectures (e.g., transformer networks) to model the dynamics of individual roles and semantic dependencies between individuals and groups. However, they rely solely on implicit pattern recognition from visual features and struggle with contextual reasoning and explainability. In this work, we propose LIR-GAD, a novel framework of language-instructed reasoning for GAD via Multimodal Large Language Model (MLLM). Our approach expand the original vocabulary of MLLM by introducing an activity-level token and multiple cluster-specific tokens. We process video frames alongside two specially designed tokens and language instructions, which are then integrated into the MLLM. The pretrained commonsense knowledge embedded in the MLLM enables the token and tokens to effectively capture the semantic information of collective activities and learn distinct representational features of different groups, respectively. Also, we introduce a multi-label classification loss to further enhance the token’s ability to learn discriminative semantic representations. Then, we design a Multimodal Dual-Alignment Fusion (MDAF) module that integrates MLLM’s hidden embeddings corresponding to the designed tokens with visual features, significantly enhancing the performance of GAD. Both quantitative and qualitative experiments demonstrate the superior performance of our proposed method in GAD taks.

Method: Proposes a dual-branch diffusion framework with: 1) BSN-based diffusion branch generating semi-clean images, and 2) conventional diffusion branch capturing underlying noise distributions. Uses BSN-based branch to guide sampling process to preserve local details while capturing noise structure.

Result: Extensive experiments on SIDD and DND datasets demonstrate state-of-the-art performance in self-supervised real-world image denoising.

Conclusion: BSGD establishes an effective self-supervised solution for real-world denoising, successfully combining the strengths of blind-spot networks and diffusion models while overcoming their individual limitations.

Abstract: In this work, we present Blind-Spot Guided Diffusion, a novel self-supervised framework for real-world image denoising. Our approach addresses two major challenges: the limitations of blind-spot networks (BSNs), which often sacrifice local detail and introduce pixel discontinuities due to spatial independence assumptions, and the difficulty of adapting diffusion models to self-supervised denoising. We propose a dual-branch diffusion framework that combines a BSN-based diffusion branch, generating semi-clean images, with a conventional diffusion branch that captures underlying noise distributions. To enable effective training without paired data, we use the BSN-based branch to guide the sampling process, capturing noise structure while preserving local details. Extensive experiments on the SIDD and DND datasets demonstrate state-of-the-art performance, establishing our method as a highly effective self-supervised solution for real-world denoising. Code and pre-trained models are released at: https://github.com/Sumching/BSGD.

Method: Proposes AdaSports-Traj with Role- and Domain-Aware Adapter to conditionally adjust latent representations, and Hierarchical Contrastive Learning to separately supervise role-sensitive and domain-aware representations for disentangled latent structures.

Result: Experiments on Basketball-U, Football-U, and Soccer-U datasets show strong performance in both unified and cross-domain trajectory prediction settings.

Conclusion: The adaptive design effectively addresses intra-domain and inter-domain distribution discrepancies in sports trajectory prediction.

Abstract: Trajectory prediction in multi-agent sports scenarios is inherently challenging due to the structural heterogeneity across agent roles (e.g., players vs. ball) and dynamic distribution gaps across different sports domains. Existing unified frameworks often fail to capture these structured distributional shifts, resulting in suboptimal generalization across roles and domains. We propose AdaSports-Traj, an adaptive trajectory modeling framework that explicitly addresses both intra-domain and inter-domain distribution discrepancies in sports. At its core, AdaSports-Traj incorporates a Role- and Domain-Aware Adapter to conditionally adjust latent representations based on agent identity and domain context. Additionally, we introduce a Hierarchical Contrastive Learning objective, which separately supervises role-sensitive and domain-aware representations to encourage disentangled latent structures without introducing optimization conflict. Experiments on three diverse sports datasets, Basketball-U, Football-U, and Soccer-U, demonstrate the effectiveness of our adaptive design, achieving strong performance in both unified and cross-domain trajectory prediction settings.

Method: The framework takes point clouds and associated 2D images as input, enriches 3D points with 2D image features, uses a 3D encoder for context fusion, and performs cross-attention between 3D anchor box queries and 2D object queries from pre-trained models to avoid memory issues while preserving 2D knowledge.

Result: SegDINO3D achieves state-of-the-art performance on ScanNetV2 and ScanNet200 benchmarks, with significant improvements of +8.7 and +6.8 mAP on ScanNet200 validation and test sets respectively.

Conclusion: The method demonstrates superior 3D instance segmentation by effectively leveraging 2D representations, with box queries enabling more precise cross-attention and achieving substantial performance gains over prior methods.

Abstract: In this paper, we present SegDINO3D, a novel Transformer encoder-decoder framework for 3D instance segmentation. As 3D training data is generally not as sufficient as 2D training images, SegDINO3D is designed to fully leverage 2D representation from a pre-trained 2D detection model, including both image-level and object-level features, for improving 3D representation. SegDINO3D takes both a point cloud and its associated 2D images as input. In the encoder stage, it first enriches each 3D point by retrieving 2D image features from its corresponding image views and then leverages a 3D encoder for 3D context fusion. In the decoder stage, it formulates 3D object queries as 3D anchor boxes and performs cross-attention from 3D queries to 2D object queries obtained from 2D images using the 2D detection model. These 2D object queries serve as a compact object-level representation of 2D images, effectively avoiding the challenge of keeping thousands of image feature maps in the memory while faithfully preserving the knowledge of the pre-trained 2D model. The introducing of 3D box queries also enables the model to modulate cross-attention using the predicted boxes for more precise querying. SegDINO3D achieves the state-of-the-art performance on the ScanNetV2 and ScanNet200 3D instance segmentation benchmarks. Notably, on the challenging ScanNet200 dataset, SegDINO3D significantly outperforms prior methods by +8.7 and +6.8 mAP on the validation and hidden test sets, respectively, demonstrating its superiority.

Method: Proposes RadarGaussianDet3D with: 1) Point Gaussian Encoder (PGE) that transforms points into Gaussian primitives using 3D Gaussian Splatting for denser BEV feature maps, and 2) Box Gaussian Loss (BGL) that converts bounding boxes into 3D Gaussian distributions for more comprehensive optimization.

Result: Extensive experiments on TJ4DRadSet and View-of-Delft datasets demonstrate state-of-the-art detection accuracy with substantially faster inference speed compared to existing methods.

Conclusion: RadarGaussianDet3D shows great potential for real-time deployment in autonomous driving by addressing key limitations of current 4D radar detectors through Gaussian-based representations.

Abstract: 4D automotive radars have gained increasing attention for autonomous driving due to their low cost, robustness, and inherent velocity measurement capability. However, existing 4D radar-based 3D detectors rely heavily on pillar encoders for BEV feature extraction, where each point contributes to only a single BEV grid, resulting in sparse feature maps and degraded representation quality. In addition, they also optimize bounding box attributes independently, leading to sub-optimal detection accuracy. Moreover, their inference speed, while sufficient for high-end GPUs, may fail to meet the real-time requirement on vehicle-mounted embedded devices. To overcome these limitations, an efficient and effective Gaussian-based 3D detector, namely RadarGaussianDet3D is introduced, leveraging Gaussian primitives and distributions as intermediate representations for radar points and bounding boxes. In RadarGaussianDet3D, a novel Point Gaussian Encoder (PGE) is designed to transform each point into a Gaussian primitive after feature aggregation and employs the 3D Gaussian Splatting (3DGS) technique for BEV rasterization, yielding denser feature maps. PGE exhibits exceptionally low latency, owing to the optimized algorithm for point feature aggregation and fast rendering of 3DGS. In addition, a new Box Gaussian Loss (BGL) is proposed, which converts bounding boxes into 3D Gaussian distributions and measures their distance to enable more comprehensive and consistent optimization. Extensive experiments on TJ4DRadSet and View-of-Delft demonstrate that RadarGaussianDet3D achieves state-of-the-art detection accuracy while delivering substantially faster inference, highlighting its potential for real-time deployment in autonomous driving.

Method: The paper systematically investigates every component in MRM development pipeline including reward modeling paradigms, reward head architecture, training strategies, data curation, backbone model and scale, and ensemble methods. Based on these insights, they introduce BaseReward built on Qwen2.5-VL backbone with optimized two-layer reward head trained on curated multimodal and text-only preference data.

Result: BaseReward establishes new SOTA on major benchmarks (MM-RLHF-Reward Bench, VL-Reward Bench, Multimodal Reward Bench) and successfully enhances MLLM performance across perception, reasoning, and conversational tasks when integrated into real-world reinforcement learning pipelines.

Conclusion: This work not only delivers a top-tier MRM but provides the community with a clear, empirically-backed guide for developing robust reward models for next-generation MLLMs.

Abstract: The rapid advancement of Multimodal Large Language Models (MLLMs) has made aligning them with human preferences a critical challenge. Reward Models (RMs) are a core technology for achieving this goal, but a systematic guide for building state-of-the-art Multimodal Reward Models (MRMs) is currently lacking in both academia and industry. Through exhaustive experimental analysis, this paper aims to provide a clear ``recipe’’ for constructing high-performance MRMs. We systematically investigate every crucial component in the MRM development pipeline, including \textit{reward modeling paradigms} (e.g., Naive-RM, Critic-based RM, and Generative RM), \textit{reward head architecture}, \textit{training strategies}, \textit{data curation} (covering over ten multimodal and text-only preference datasets), \textit{backbone model} and \textit{model scale}, and \textit{ensemble methods}. Based on these experimental insights, we introduce \textbf{BaseReward}, a powerful and efficient baseline for multimodal reward modeling. BaseReward adopts a simple yet effective architecture, built upon a {Qwen2.5-VL} backbone, featuring an optimized two-layer reward head, and is trained on a carefully curated mixture of high-quality multimodal and text-only preference data. Our results show that BaseReward establishes a new SOTA on major benchmarks such as MM-RLHF-Reward Bench, VL-Reward Bench, and Multimodal Reward Bench, outperforming previous models. Furthermore, to validate its practical utility beyond static benchmarks, we integrate BaseReward into a real-world reinforcement learning pipeline, successfully enhancing an MLLM’s performance across various perception, reasoning, and conversational tasks. This work not only delivers a top-tier MRM but, more importantly, provides the community with a clear, empirically-backed guide for developing robust reward models for the next generation of MLLMs.

Method: A hybrid approach combining feed-forward prediction to infer scene parameters and differentiable rendering within an analysis-by-synthesis framework to refine parameter estimates.

Result: The method effectively recovers object pose given an untextured 3D model in both simulations and real-world captures, showing promising results for parametric scenes with strong priors.

Conclusion: The approach demonstrates feasibility of using distributed sparse measurements from time-of-flight sensors for 3D geometry recovery, with potential applications in simple parametric scene reconstruction.

Abstract: We aim to recover the geometry of 3D parametric scenes using very few depth measurements from low-cost, commercially available time-of-flight sensors. These sensors offer very low spatial resolution (i.e., a single pixel), but image a wide field-of-view per pixel and capture detailed time-of-flight data in the form of time-resolved photon counts. This time-of-flight data encodes rich scene information and thus enables recovery of simple scenes from sparse measurements. We investigate the feasibility of using a distributed set of few measurements (e.g., as few as 15 pixels) to recover the geometry of simple parametric scenes with a strong prior, such as estimating the 6D pose of a known object. To achieve this, we design a method that utilizes both feed-forward prediction to infer scene parameters, and differentiable rendering within an analysis-by-synthesis framework to refine the scene parameter estimate. We develop hardware prototypes and demonstrate that our method effectively recovers object pose given an untextured 3D model in both simulations and controlled real-world captures, and show promising initial results for other parametric scenes. We additionally conduct experiments to explore the limits and capabilities of our imaging solution.

Method: Developed AcT2I benchmark for systematic evaluation, then created a training-free knowledge distillation technique using Large Language Models to enhance prompts by incorporating dense information across three dimensions (with temporal details being most impactful).

Result: Leading T2I models perform poorly on AcT2I, but the proposed method significantly improves image generation accuracy by 72% through prompt enhancement with temporal details.

Conclusion: Current T2I methods have limitations in generating images requiring complex reasoning, but systematic integration of linguistic knowledge can notably advance generation of nuanced and contextually accurate images.

Abstract: Text-to-Image (T2I) models have recently achieved remarkable success in generating images from textual descriptions. However, challenges still persist in accurately rendering complex scenes where actions and interactions form the primary semantic focus. Our key observation in this work is that T2I models frequently struggle to capture nuanced and often implicit attributes inherent in action depiction, leading to generating images that lack key contextual details. To enable systematic evaluation, we introduce AcT2I, a benchmark designed to evaluate the performance of T2I models in generating images from action-centric prompts. We experimentally validate that leading T2I models do not fare well on AcT2I. We further hypothesize that this shortcoming arises from the incomplete representation of the inherent attributes and contextual dependencies in the training corpora of existing T2I models. We build upon this by developing a training-free, knowledge distillation technique utilizing Large Language Models to address this limitation. Specifically, we enhance prompts by incorporating dense information across three dimensions, observing that injecting prompts with temporal details significantly improves image generation accuracy, with our best model achieving an increase of 72%. Our findings highlight the limitations of current T2I methods in generating images that require complex reasoning and demonstrate that integrating linguistic knowledge in a systematic way can notably advance the generation of nuanced and contextually accurate images.

Method: Proposed Sycophantic Reflective Tuning (SRT) that enables MLLMs to engage in reflective reasoning to distinguish between misleading and corrective instructions before drawing conclusions, avoiding the trade-off of naive supervised fine-tuning.

Result: SRT significantly reduces sycophantic behavior toward misleading instructions without causing excessive stubbornness when receiving corrective instructions.

Conclusion: The sycophantic modality gap is a critical issue in MLLMs, and SRT provides an effective solution that balances resistance to misleading instructions while maintaining responsiveness to valid corrections.

Abstract: Multimodal large language models (MLLMs) have demonstrated extraordinary capabilities in conducting conversations based on image inputs. However, we observe that MLLMs exhibit a pronounced form of visual sycophantic behavior. While similar behavior has also been noted in text-based large language models (LLMs), it becomes significantly more prominent when MLLMs process image inputs. We refer to this phenomenon as the “sycophantic modality gap.” To better understand this issue, we further analyze the factors that contribute to the exacerbation of this gap. To mitigate the visual sycophantic behavior, we first experiment with naive supervised fine-tuning to help the MLLM resist misleading instructions from the user. However, we find that this approach also makes the MLLM overly resistant to corrective instructions (i.e., stubborn even if it is wrong). To alleviate this trade-off, we propose Sycophantic Reflective Tuning (SRT), which enables the MLLM to engage in reflective reasoning, allowing it to determine whether a user’s instruction is misleading or corrective before drawing a conclusion. After applying SRT, we observe a significant reduction in sycophantic behavior toward misleading instructions, without resulting in excessive stubbornness when receiving corrective instructions.

Method: Uses hierarchical self-supervised compensation (HSSC) across three levels: modality reconstruction with hybrid shuffled-masking augmentation, modality-invariant contrastive learning, and lightweight reverse attention adapter. Fine-tuned under hybrid consistency constraint.

Result: Significantly outperforms state-of-the-art methods in diverse missing modality scenarios across MRI brain tumor segmentation, RGB-D semantic segmentation, and RGB-D/T salient object segmentation tasks.

Conclusion: UniMRSeg provides an effective unified solution for multi-modal segmentation with missing modalities, reducing deployment complexity while maintaining high performance across various real-world scenarios.

Abstract: Multi-modal image segmentation faces real-world deployment challenges from incomplete/corrupted modalities degrading performance. While existing methods address training-inference modality gaps via specialized per-combination models, they introduce high deployment costs by requiring exhaustive model subsets and model-modality matching. In this work, we propose a unified modality-relax segmentation network (UniMRSeg) through hierarchical self-supervised compensation (HSSC). Our approach hierarchically bridges representation gaps between complete and incomplete modalities across input, feature and output levels. % First, we adopt modality reconstruction with the hybrid shuffled-masking augmentation, encouraging the model to learn the intrinsic modality characteristics and generate meaningful representations for missing modalities through cross-modal fusion. % Next, modality-invariant contrastive learning implicitly compensates the feature space distance among incomplete-complete modality pairs. Furthermore, the proposed lightweight reverse attention adapter explicitly compensates for the weak perceptual semantics in the frozen encoder. Last, UniMRSeg is fine-tuned under the hybrid consistency constraint to ensure stable prediction under all modality combinations without large performance fluctuations. Without bells and whistles, UniMRSeg significantly outperforms the state-of-the-art methods under diverse missing modality scenarios on MRI-based brain tumor segmentation, RGB-D semantic segmentation, RGB-D/T salient object segmentation. The code will be released at https://github.com/Xiaoqi-Zhao-DLUT/UniMRSeg.

Method: DSEG-LIME introduces: 1) data-driven segmentation using foundation models to generate human-recognized features, and 2) user-steered granularity control in hierarchical segmentation through composition.

Result: DSEG-LIME outperforms standard LIME on multiple XAI metrics for pre-trained ImageNet models and improves explanation alignment with human-recognized concepts.

Conclusion: The framework successfully addresses LIME’s segmentation limitations by integrating foundation models and user-controlled granularity, enhancing explanation quality and human interpretability.

Abstract: LIME (Local Interpretable Model-agnostic Explanations) is a popular XAI framework for unraveling decision-making processes in vision machine-learning models. The technique utilizes image segmentation methods to identify fixed regions for calculating feature importance scores as explanations. Therefore, poor segmentation can weaken the explanation and reduce the importance of segments, ultimately affecting the overall clarity of interpretation. To address these challenges, we introduce the DSEG-LIME (Data-Driven Segmentation LIME) framework, featuring: i) a data-driven segmentation for human-recognized feature generation by foundation model integration, and ii) a user-steered granularity in the hierarchical segmentation procedure through composition. Our findings demonstrate that DSEG outperforms on several XAI metrics on pre-trained ImageNet models and improves the alignment of explanations with human-recognized concepts. The code is available under: https://github. com/patrick-knab/DSEG-LIME

Method: Proposes an avoid-shortcut learning framework using well-designed shortcut primes based on Neural Collapse structure, encouraging models to skip simple shortcuts and capture intrinsic correlations without additional training complexity.

Result: Experimental results show the method induces better convergence properties during training and achieves state-of-the-art generalization performance on both synthetic and real-world biased datasets.

Conclusion: The proposed framework effectively prevents shortcut learning in biased datasets by leveraging Neural Collapse principles, leading to improved model generalization without increasing training complexity.

Abstract: Recent studies have noted an intriguing phenomenon termed Neural Collapse, that is, when the neural networks establish the right correlation between feature spaces and the training targets, their last-layer features, together with the classifier weights, will collapse into a stable and symmetric structure. In this paper, we extend the investigation of Neural Collapse to the biased datasets with imbalanced attributes. We observe that models will easily fall into the pitfall of shortcut learning and form a biased, non-collapsed feature space at the early period of training, which is hard to reverse and limits the generalization capability. To tackle the root cause of biased classification, we follow the recent inspiration of prime training, and propose an avoid-shortcut learning framework without additional training complexity. With well-designed shortcut primes based on Neural Collapse structure, the models are encouraged to skip the pursuit of simple shortcuts and naturally capture the intrinsic correlations. Experimental results demonstrate that our method induces better convergence properties during training, and achieves state-of-the-art generalization performance on both synthetic and real-world biased datasets. Our code is available at https://github.com/RachelWolowitz/Navigate-beyond-Shortcuts/tree/main.

Method: The authors propose a re-calibration model that takes LiDAR point cloud, camera image, and initial calibration matrix as inputs, generating re-calibrated bias through semantic segmentation guidance and a tailored loss function design. The model works with existing detection algorithms.

Result: The paper systematically evaluates EPNet++ framework sensitivity and proves that even slight calibration bias seriously reduces performance. The proposed re-calibration model enhances both robustness against calibration bias and overall object detection performance.

Conclusion: This approach establishes a foundational methodology for maintaining reliability in multi-modal perception systems under real-world calibration uncertainties, providing a solution to calibration dependency issues in autonomous driving.

Abstract: Multi-modal object detection in autonomous driving has achieved great breakthroughs due to the usage of fusing complementary information from different sensors. The calibration in fusion between sensors such as LiDAR and camera was always supposed to be precise in previous work. However, in reality, calibration matrices are fixed when the vehicles leave the factory, but mechanical vibration, road bumps, and data lags may cause calibration bias. As there is relatively limited research on the impact of calibration on fusion detection performance, multi-sensor detection methods with flexible calibration dependency have remained a key objective. In this paper, we systematically evaluate the sensitivity of the SOTA EPNet++ detection framework and prove that even slight bias on calibration can reduce the performance seriously. To address this vulnerability, we propose a re-calibration model to re-calibrate the misalignment in detection tasks. This model integrates LiDAR point cloud, camera image, and initial calibration matrix as inputs, generating re-calibrated bias through semantic segmentation guidance and a tailored loss function design. The re-calibration model can operate with existing detection algorithms, enhancing both robustness against calibration bias and overall object detection performance. Our approach establishes a foundational methodology for maintaining reliability in multi-modal perception systems under real-world calibration uncertainties.

Method: The methodology has two elements: (1) determining a visibility threshold in a common subjective representation using psychophysics, and (2) transducing metric distance values to this common representation. The approach uses subjective ratings from existing image quality databases.

Result: Testing established metrics revealed that none of them exhibit human-like invisibility thresholds, indicating they fail to capture human vision’s invariance properties to natural transformations.

Conclusion: Tuning models exclusively to predict generic distortion visibility may disregard other human vision properties like invariances and invisibility thresholds. The method provides a framework to test metrics for human-like behavior to affine transformations.

Abstract: Subjective image quality metrics are usually evaluated according to the correlation with human opinion in databases with distortions that may appear in digital media. However, these oversee affine transformations which may represent better the changes in the images actually happening in natural conditions. Humans can be particularly invariant to these natural transformations, as opposed to the digital ones. In this work, we propose a methodology to evaluate any image quality metric by assessing their invariance to affine transformations, specifically: rotation, translation, scaling, and changes in spectral illumination. Here, invariance refers to the fact that certain distances should be neglected if their values are below a threshold. This is what we call invisibility threshold of a metric. Our methodology consists of two elements: (1) the determination of a visibility threshold in a subjective representation common to every metric, and (2) a transduction from the distance values of the metric and this common representation. This common representation is based on subjective ratings of readily available image quality databases. We determine the threshold in such common representation (the first element) using accurate psychophysics. Then, the transduction (the second element) can be trivially fitted for any metric: with the provided threshold extension of the method to any metric is straightforward. We test our methodology with some well-established metrics and find that none of them show human-like invisibility thresholds. This means that tuning the models exclusively to predict the visibility of generic distortions may disregard other properties of human vision as for instance invariances or invisibility thresholds. The data and code are publicly available to test other metrics.

Method: Conditionally factorizes the task: 1) Uses vision-language models (Llama3.2) to predict discrete base structure with semantics, 2) Proposes TrAssembler to predict continuous attributes conditioned on the discrete structure. Constructs an annotated CAD dataset from ShapeNet for training.

Result: The approach demonstrates significant first steps towards CAD-ifying images in the wild, with code and data made available.

Conclusion: The proposed factorization method and dataset represent important progress in bridging the gap between image inputs and precise CAD model outputs for reverse engineering applications.

Abstract: Reverse engineering 3D computer-aided design (CAD) models from images is an important task for many downstream applications including interactive editing, manufacturing, architecture, robotics, etc. The difficulty of the task lies in vast representational disparities between the CAD output and the image input. CAD models are precise, programmatic constructs that involve sequential operations combining discrete command structure with continuous attributes, making it challenging to learn and optimize in an end-to-end fashion. Concurrently, input images introduce inherent challenges such as photometric variability and sensor noise, complicating the reverse engineering process. In this work, we introduce a novel approach that conditionally factorizes the task into two sub-problems. First, we leverage vision-language foundation models (VLMs), a finetuned Llama3.2, to predict the global discrete base structure with semantic information. Second, we propose TrAssembler that, conditioned on the discrete structure with semantics, predicts the continuous attribute values. To support the training of our TrAssembler, we further constructed an annotated CAD dataset of common objects from ShapeNet. Putting all together, our approach and data demonstrate significant first steps towards CAD-ifying images in the wild. Code and data can be found in https://github.com/qq456cvb/Img2CAD.

Method: Combines spatiotemporal sequence modeling via LSTM networks with subject-invariant feature engineering and normalization, outperforming Transformers, TCNs, and CNNs especially with limited/noisy gaze data.

Result: Achieves mean absolute error of 9.1 cm across three benchmark datasets using LOOCV and cross-dataset validation, showing strong generalization without calibration.

Conclusion: FOVAL’s scalability and accuracy make it highly suitable for real-world deployment, with analysis showing robustness to inter-subject variability and domain shifts.

Abstract: Accurate fixation depth estimation is essential for applications in extended reality (XR), robotics, and human-computer interaction. However, current methods heavily depend on user-specific calibration, which limits their scalability and usability. We introduce FOVAL, a robust calibration-free approach that combines spatiotemporal sequence modelling via Long Short-Term Memory (LSTM) networks with subject-invariant feature engineering and normalisation. Compared to Transformers, Temporal Convolutional Networks (TCNs), and CNNs, FOVAL achieves superior performance, particularly in scenarios with limited and noisy gaze data. Evaluations across three benchmark datasets using Leave-One-Out Cross-Validation (LOOCV) and cross-dataset validation show a mean absolute error (MAE) of 9.1 cm and strong generalisation without calibration. We further analyse inter-subject variability and domain shifts, providing insight into model robustness and adaptation. FOVAL’s scalability and accuracy make it highly suitable for real-world deployment.

Method: Combo uses a two-stage approach: pre-training on fixed identity with neutral emotion, then fine-tuning with customizable conditions using X-Adapter for parameter efficiency. It employs DU-Trans transformer that divides into face/body branches and unites them for joint bi-directional distribution learning.

Result: Evaluated on BEAT2 and SHOW datasets, Combo demonstrates high-quality motion generation and efficient identity/emotion transfer capabilities.

Conclusion: The proposed Combo framework effectively addresses the MIMO challenges in co-speech holistic 3D human motion generation through its tailored input handling and output coordination designs, achieving both high-quality generation and efficient customization.

Abstract: In this paper, we propose a novel framework, Combo, for harmonious co-speech holistic 3D human motion generation and efficient customizable adaption. In particular, we identify that one fundamental challenge as the multiple-input-multiple-output (MIMO) nature of the generative model of interest. More concretely, on the input end, the model typically consumes both speech signals and character guidance (e.g., identity and emotion), which not only poses challenge on learning capacity but also hinders further adaptation to varying guidance; on the output end, holistic human motions mainly consist of facial expressions and body movements, which are inherently correlated but non-trivial to coordinate in current data-driven generation process. In response to the above challenge, we propose tailored designs to both ends. For the former, we propose to pre-train on data regarding a fixed identity with neutral emotion, and defer the incorporation of customizable conditions (identity and emotion) to fine-tuning stage, which is boosted by our novel X-Adapter for parameter-efficient fine-tuning. For the latter, we propose a simple yet effective transformer design, DU-Trans, which first divides into two branches to learn individual features of face expression and body movements, and then unites those to learn a joint bi-directional distribution and directly predicts combined coefficients. Evaluated on BEAT2 and SHOW datasets, Combo is highly effective in generating high-quality motions but also efficient in transferring identity and emotion. Project website: \href{https://xc-csc101.github.io/combo/}{Combo}.

Method: Proposed CrackSCF network uses lightweight convolutional blocks (LRDS) to capture local patterns, a Long-range Dependency Extractor (LDE) for global dependencies, and a Staircase Cascaded Fusion Module (SCFM) to intelligently unify local and global features.

Result: CrackSCF achieved 0.8382 F1 score and 0.8473 mIoU on the TUT dataset with only 4.79M parameters, consistently outperforming existing methods and demonstrating greater robustness against complex background noise across multiple datasets.

Conclusion: The proposed CrackSCF method successfully addresses the limitations of existing crack segmentation approaches by providing robust segmentation with exceptional computational efficiency, making it suitable for practical deployment on edge devices.

Abstract: Accurately segmenting structural cracks at the pixel level remains a major hurdle, as existing methods fail to integrate local textures with pixel dependencies, often leading to fragmented and incomplete predictions. Moreover, their high parameter counts and substantial computational demands hinder practical deployment on resource-constrained edge devices. To address these challenges, we propose CrackSCF, a Lightweight Cascaded Fusion Crack Segmentation Network designed to achieve robust crack segmentation with exceptional computational efficiency. We design a lightweight convolutional block (LRDS) to replace all standard convolutions. This approach efficiently captures local patterns while operating with a minimal computational footprint. For a holistic perception of crack structures, a lightweight Long-range Dependency Extractor (LDE) captures global dependencies. These are then intelligently unified with local patterns by our Staircase Cascaded Fusion Module (SCFM), ensuring the final segmentation maps are both seamless in continuity and rich in fine-grained detail. To comprehensively evaluate our method, we created the challenging TUT benchmark dataset and evaluated it alongside five other public datasets. The experimental results show that the CrackSCF method consistently outperforms the existing methods, and it demonstrates greater robustness in dealing with complex background noise. On the TUT dataset, CrackSCF achieved 0.8382 on F1 score and 0.8473 on mIoU, and it only required 4.79M parameters.

Method: Proposes DAOcc framework with 3D object detection supervision to guide occupancy prediction, BEV View Range Extension strategy to compensate for lower resolution, and uses ResNet-50 backbone with 256*704 input resolution for practical deployment.

Result: Achieves new SOTA on Occ3D-nuScenes and Occ3D-Waymo benchmarks, outperforming previous methods significantly. With TensorRT optimization, reaches 104.9 FPS while maintaining 54.2 mIoU on RTX 4090 GPU.

Conclusion: DAOcc demonstrates that effective supervision strategies and practical design choices can achieve superior performance without relying on high-resolution inputs or complex networks, enabling real-time deployment in autonomous driving applications.

Abstract: Multi-sensor fusion significantly enhances the accuracy and robustness of 3D semantic occupancy prediction, which is crucial for autonomous driving and robotics. However, most existing approaches depend on high-resolution images and complex networks to achieve top performance, hindering their deployment in practical scenarios. Moreover, current multi-sensor fusion approaches mainly focus on improving feature fusion while largely neglecting effective supervision strategies for those features. To address these issues, we propose DAOcc, a novel multi-modal occupancy prediction framework that leverages 3D object detection supervision to assist in achieving superior performance, while using a deployment-friendly image backbone and practical input resolution. In addition, we introduce a BEV View Range Extension strategy to mitigate performance degradation caused by lower image resolution. Extensive experiments demonstrate that DAOcc achieves new state-of-the-art results on both the Occ3D-nuScenes and Occ3D-Waymo benchmarks, and outperforms previous state-of-the-art methods by a significant margin using only a ResNet-50 backbone and 256*704 input resolution. With TensorRT optimization, DAOcc reaches 104.9 FPS while maintaining 54.2 mIoU on an NVIDIA RTX 4090 GPU. Code is available at https://github.com/AlphaPlusTT/DAOcc.

Method: A two-stage diffusion-based framework: 1) Region Proposal stage to identify problematic areas, and 2) Depth Inpainting stage that dynamically analyzes optical and geometric depth loss to automatically inpaint missing depth information.

Result: Comprehensive experiments demonstrate that DITR is highly effective for depth inpainting tasks on transparent and reflective objects with robust adaptability.

Conclusion: The proposed DITR framework successfully addresses the challenge of depth acquisition for transparent and reflective objects, providing an effective solution for 3D imaging applications involving such materials.

Abstract: Transparent and reflective objects, which are common in our everyday lives, present a significant challenge to 3D imaging techniques due to their unique visual and optical properties. Faced with these types of objects, RGB-D cameras fail to capture the real depth value with their accurate spatial information. To address this issue, we propose DITR, a diffusion-based Depth Inpainting framework specifically designed for Transparent and Reflective objects. This network consists of two stages, including a Region Proposal stage and a Depth Inpainting stage. DITR dynamically analyzes the optical and geometric depth loss and inpaints them automatically. Furthermore, comprehensive experimental results demonstrate that DITR is highly effective in depth inpainting tasks of transparent and reflective objects with robust adaptability.

Method: Approximates true posterior with variational distribution using categorical distributions and continuous relaxation. Employs star-shaped noise process to mitigate drawbacks of traditional discrete diffusion models with absorbing states.

Result: The method performs comparably to continuous diffusion techniques while achieving lower GPU memory consumption.

Conclusion: The proposed approach successfully enables discrete diffusion models to solve linear inverse problems in continuous spaces, offering memory-efficient performance comparable to continuous methods.

Abstract: Recent literature has effectively leveraged diffusion models trained on continuous variables as priors for solving inverse problems. Notably, discrete diffusion models with discrete latent codes have shown strong performance, particularly in modalities suited for discrete compressed representations, such as image and motion generation. However, their discrete and non-differentiable nature has limited their application to inverse problems formulated in continuous spaces. This paper presents a novel method for addressing linear inverse problems by leveraging generative models based on discrete diffusion as priors. We overcome these limitations by approximating the true posterior distribution with a variational distribution constructed from categorical distributions and continuous relaxation techniques. Furthermore, we employ a star-shaped noise process to mitigate the drawbacks of traditional discrete diffusion models with absorbing states, demonstrating that our method performs comparably to continuous diffusion techniques with a lower GPU memory consumption. Our code is available at https://github.com/sony/g2d2.

Method: Developed an extended SMILES encoding rule to annotate training data, created MolParser-7M (largest annotated molecular image dataset), used active learning to incorporate real-world data from patents/literature, and trained an end-to-end molecular image captioning model using curriculum learning.

Result: MolParser significantly outperforms both classical and learning-based OCSR methods across most scenarios, demonstrating superior accuracy in recognizing chemical structures from diverse real-world documents.

Conclusion: The method shows strong potential for broader downstream applications in chemistry, biology, and pharmaceuticals. The dataset is publicly available to support further research.

Abstract: In recent decades, chemistry publications and patents have increased rapidly. A significant portion of key information is embedded in molecular structure figures, complicating large-scale literature searches and limiting the application of large language models in fields such as biology, chemistry, and pharmaceuticals. The automatic extraction of precise chemical structures is of critical importance. However, the presence of numerous Markush structures in real-world documents, along with variations in molecular image quality, drawing styles, and noise, significantly limits the performance of existing optical chemical structure recognition (OCSR) methods. We present MolParser, a novel end-to-end OCSR method that efficiently and accurately recognizes chemical structures from real-world documents, including difficult Markush structure. We use a extended SMILES encoding rule to annotate our training dataset. Under this rule, we build MolParser-7M, the largest annotated molecular image dataset to our knowledge. While utilizing a large amount of synthetic data, we employed active learning methods to incorporate substantial in-the-wild data, specifically samples cropped from real patents and scientific literature, into the training process. We trained an end-to-end molecular image captioning model, MolParser, using a curriculum learning approach. MolParser significantly outperforms classical and learning-based methods across most scenarios, with potential for broader downstream applications. The dataset is publicly available in huggingface.

Method: CADSpotting represents primitives through densely sampled points with only coordinate attributes using a unified 3D point cloud model. It introduces Sliding Window Aggregation (SWA) combining weighted voting and Non-Maximum Suppression for accurate segmentation in large drawings.

Result: Experiments on FloorPlanCAD and the new LS-CAD dataset (45 large floorplans) show CADSpotting significantly outperforms existing methods. It also enables automated parametric 3D interior reconstruction from raw CAD inputs.

Conclusion: CADSpotting provides an effective solution for panoptic symbol spotting in large-scale CAD drawings and demonstrates practical value for automated 3D reconstruction applications.

Abstract: We introduce CADSpotting, an effective method for panoptic symbol spotting in large-scale architectural CAD drawings. Existing approaches often struggle with symbol diversity, scale variations, and overlapping elements in CAD designs, and typically rely on additional features (e.g., primitive types or graphical layers) to improve performance. CADSpotting overcomes these challenges by representing primitives through densely sampled points with only coordinate attributes, using a unified 3D point cloud model for robust feature learning. To enable accurate segmentation in large drawings, we further propose a novel Sliding Window Aggregation (SWA) technique that combines weighted voting and Non-Maximum Suppression (NMS). Moreover, we introduce LS-CAD, a new large-scale dataset comprising 45 finely annotated floorplans, each covering approximately 1,000 $m^2$, significantly larger than prior benchmarks. LS-CAD will be publicly released to support future research. Experiments on FloorPlanCAD and LS-CAD demonstrate that CADSpotting significantly outperforms existing methods. We also showcase its practical value by enabling automated parametric 3D interior reconstruction directly from raw CAD inputs.

Method: The authors introduce Near-Field Lighting Bundle Adjustment Loss (NFL-BA) that explicitly incorporates near-field lighting modeling into Bundle Adjustment, compatible with both implicit and explicit scene representations in neural rendering-based SLAM systems.

Result: NFL-BA achieves significant improvements: 37% better camera tracking for MonoGS and 14% for EndoGS on the C3VD colonoscopy dataset, plus improved performance in indoor scenes with phone flashlights.

Conclusion: NFL-BA enables state-of-the-art SLAM performance in challenging near-field lighting conditions, with potential applications in medical endoscopy, subterranean robotics, and search & rescue operations.

Abstract: Simultaneous Localization and Mapping (SLAM) systems typically assume static, distant illumination; however, many real-world scenarios, such as endoscopy, subterranean robotics, and search & rescue in collapsed environments, require agents to operate with a co-located light and camera in the absence of external lighting. In such cases, dynamic near-field lighting introduces strong, view-dependent shading that significantly degrades SLAM performance. We introduce Near-Field Lighting Bundle Adjustment Loss (NFL-BA) which explicitly models near-field lighting as a part of Bundle Adjustment loss and enables better performance for scenes captured with dynamic lighting. NFL-BA can be integrated into neural rendering-based SLAM systems with implicit or explicit scene representations. Our evaluations mainly focus on endoscopy procedure where SLAM can enable autonomous navigation, guidance to unsurveyed regions, blindspot detections, and 3D visualizations, which can significantly improve patient outcomes and endoscopy experience for both physicians and patients. Replacing Photometric Bundle Adjustment loss of SLAM systems with NFL-BA leads to significant improvement in camera tracking, 37% for MonoGS and 14% for EndoGS, and leads to state-of-the-art camera tracking and mapping performance on the C3VD colonoscopy dataset. Further evaluation on indoor scenes captured with phone camera with flashlight turned on, also demonstrate significant improvement in SLAM performance due to NFL-BA. See results at https://asdunnbe.github.io/NFL-BA/

Method: The study evaluates state-of-the-art affective computing models (facial expression recognition, text sentiment analysis, smile detection) using videos of older adults interacting with people or avatars, and introduces a novel dataset of Spanish-speaking older adults in video interviews.

Result: Analyses reveal limited agreement between human annotations and model predictions, weak consistency across modalities, and significant individual variability in emotional signals.

Conclusion: Generalized emotion perception models have shortcomings; future systems must incorporate personal variability and cultural nuances for effective emotional signal understanding in older adults.

Abstract: Understanding emotional signals in older adults is crucial for designing virtual assistants that support their well-being. However, existing affective computing models often face significant limitations: (1) limited availability of datasets representing older adults, especially in non-English-speaking populations, and (2) poor generalization of models trained on younger or homogeneous demographics. To address these gaps, this study evaluates state-of-the-art affective computing models – including facial expression recognition, text sentiment analysis, and smile detection – using videos of older adults interacting with either a person or a virtual avatar. As part of this effort, we introduce a novel dataset featuring Spanish-speaking older adults engaged in human-to-human video interviews. Through three comprehensive analyses, we investigate (1) the alignment between human-annotated labels and automatic model outputs, (2) the relationships between model outputs across different modalities, and (3) individual variations in emotional signals. Using both the Wizard of Oz (WoZ) dataset and our newly collected dataset, we uncover limited agreement between human annotations and model predictions, weak consistency across modalities, and significant variability among individuals. These findings highlight the shortcomings of generalized emotion perception models and emphasize the need of incorporating personal variability and cultural nuances into future systems.

Method: The survey systematically reviews five key areas: multimodal domain adaptation, multimodal test-time adaptation, multimodal domain generalization, domain adaptation/generalization with foundation models, and adaptation of multimodal foundation models. For each topic, it provides formal problem definitions and thorough method reviews.

Result: The survey analyzes relevant datasets and applications, highlighting open challenges and future research directions. It maintains an active repository with up-to-date literature.

Conclusion: This work establishes the first comprehensive framework for understanding multimodal adaptation and generalization, providing valuable insights for researchers working with multimodal foundation models and their applications in real-world scenarios.

Abstract: In real-world scenarios, achieving domain adaptation and generalization poses significant challenges, as models must adapt to or generalize across unknown target distributions. Extending these capabilities to unseen multimodal distributions, i.e., multimodal domain adaptation and generalization, is even more challenging due to the distinct characteristics of different modalities. Significant progress has been made over the years, with applications ranging from action recognition to semantic segmentation. Besides, the recent advent of large-scale pre-trained multimodal foundation models, such as CLIP, has inspired works leveraging these models to enhance adaptation and generalization performances or adapting them to downstream tasks. This survey provides the first comprehensive review of recent advances from traditional approaches to foundation models, covering: (1) Multimodal domain adaptation; (2) Multimodal test-time adaptation; (3) Multimodal domain generalization; (4) Domain adaptation and generalization with the help of multimodal foundation models; and (5) Adaptation of multimodal foundation models. For each topic, we formally define the problem and thoroughly review existing methods. Additionally, we analyze relevant datasets and applications, highlighting open challenges and potential future research directions. We maintain an active repository that contains up-to-date literature at https://github.com/donghao51/Awesome-Multimodal-Adaptation.

Method: Enhanced density-based UVAS framework with dense self-supervised learning for feature extraction and learned masking-invariant dense features as conditioning variables, trained on 30,000+ unlabeled 3D CT volumes.

Result: Outperforms existing UVAS methods on four test datasets (1,820 scans) and surpasses self-supervised pretraining methods in supervised fine-tuning, establishing state-of-the-art performance.

Conclusion: Screener provides a state-of-the-art foundation for pathology segmentation that eliminates the need for supervised pretraining and hand-crafted features, with code and models to be publicly released.

Abstract: Accurate detection of all pathological findings in 3D medical images remains a significant challenge, as supervised models are limited to detecting only the few pathology classes annotated in existing datasets. To address this, we frame pathology detection as an unsupervised visual anomaly segmentation (UVAS) problem, leveraging the inherent rarity of pathological patterns compared to healthy ones. We enhance the existing density-based UVAS framework with two key innovations: (1) dense self-supervised learning for feature extraction, eliminating the need for supervised pretraining, and (2) learned, masking-invariant dense features as conditioning variables, replacing hand-crafted positional encodings. Trained on over 30,000 unlabeled 3D CT volumes, our fully self-supervised model, Screener, outperforms existing UVAS methods on four large-scale test datasets comprising 1,820 scans with diverse pathologies. Furthermore, in a supervised fine-tuning setting, Screener surpasses existing self-supervised pretraining methods, establishing it as a state-of-the-art foundation for pathology segmentation. The code and pretrained models will be made publicly available.

Method: Integrated high-resolution physical model simulations with spatial and meteorological data using ST-ViT model to develop fine-scale human thermal predictions for a Texas campus testbed.

Result: The ST-ViT-powered digital twin enables efficient, data-driven insights for planners and stakeholders, supporting targeted heat mitigation strategies.

Conclusion: This campus-scale demonstration provides a foundation for future applications across broader and more diverse urban contexts, advancing climate-adaptive urban design.

Abstract: Extreme heat events, exacerbated by climate change, pose significant challenges to urban resilience and planning. This study introduces a climate-responsive digital twin framework integrating the Spatiotemporal Vision Transformer (ST-ViT) model to enhance heat stress forecasting and decision-making. Using a Texas campus as a testbed, we synthesized high-resolution physical model simulations with spatial and meteorological data to develop fine-scale human thermal predictions. The ST-ViT-powered digital twin enables efficient, data-driven insights for planners and stakeholders, supporting targeted heat mitigation strategies and advancing climate-adaptive urban design. This campus-scale demonstration offers a foundation for future applications across broader and more diverse urban contexts.

Method: Proposes Straight Consistent Trajectory (SCoT) model that balances two objectives: regulating gradient mapping to constant and ensuring trajectory consistency, creating trajectories with both straight and consistent properties.

Result: Extensive experiments demonstrate SCoT’s effectiveness and efficiency in fast sampling while maintaining quality.

Conclusion: SCoT successfully bridges the gap between consistency models and rectified flow, achieving fast sampling with straight and consistent trajectories simultaneously.

Abstract: Pre-trained diffusion models are commonly used to generate clean data (e.g., images) from random noises, effectively forming pairs of noises and corresponding clean images. Distillation on these pre-trained models can be viewed as the process of constructing advanced trajectories within the pair to accelerate sampling. For instance, consistency model distillation develops consistent projection functions to regulate trajectories, although sampling efficiency remains a concern. Rectified flow method enforces straight trajectories to enable faster sampling, yet relies on numerical ODE solvers, which may introduce approximation errors. In this work, we bridge the gap between the consistency model and the rectified flow method by proposing a Straight Consistent Trajectory~(SCoT) model. SCoT enjoys the benefits of both approaches for fast sampling, producing trajectories with consistent and straight properties simultaneously. These dual properties are strategically balanced by targeting two critical objectives: (1) regulating the gradient of SCoT’s mapping to a constant, (2) ensuring trajectory consistency. Extensive experimental results demonstrate the effectiveness and efficiency of SCoT.

Method: Develop a score model trained on large-scale RS vision-language preference data to automatically assess quality of synthetic image-text pairs, then use top-ranked data for fine-tuning CLIP and advanced VLMs.

Result: Fine-tuning with top 30% data ranked by the score model achieves superior accuracy over full-data fine-tuning and CLIP-score-based approaches, with additional improvements through RL training and BoN test-time scaling.

Conclusion: The proposed automated quality assessment framework effectively addresses the data quality challenge in RS vision-language tasks, enabling significant VLM performance improvements and providing practical applications for data filtering and model enhancement.

Abstract: Vision-Language Models (VLMs) have demonstrated great potential in interpreting remote sensing (RS) images through language-guided semantic. However, the effectiveness of these VLMs critically depends on high-quality image-text training data that captures rich semantic relationships between visual content and language descriptions. Unlike natural images, RS lacks large-scale interleaved image-text pairs from web data, making data collection challenging. While current approaches rely primarily on rule-based methods or flagship VLMs for data synthesis, a systematic framework for automated quality assessment of such synthetically generated RS vision-language data is notably absent. To fill this gap, we propose a novel score model trained on large-scale RS vision-language preference data for automated quality assessment. Our empirical results demonstrate that fine-tuning CLIP or advanced VLMs (e.g., Qwen2-VL) with the top 30% of data ranked by our score model achieves superior accuracy compared to both full-data fine-tuning and CLIP-score-based ranking approaches. Furthermore, we demonstrate applications of our scoring model for reinforcement learning (RL) training and best-of-N (BoN) test-time scaling, enabling significant improvements in VLM performance for RS tasks. Our code, model, and dataset are publicly available

Method: Created ISP-AD dataset with synthetic and real defects from factory floor, featuring small and weakly contrasted surface defects in structured patterns. Conducted experiments on unsupervised anomaly detection methods and mixed supervised training strategy combining synthetic and real defects.

Result: Even small amounts of weakly labeled real defects improve generalization. Model-free synthetic defects provide cold-start baseline, while injected real defects refine decision boundaries for unseen defect characteristics. Dataset supports unsupervised, self-supervised, and supervised approaches.

Conclusion: ISP-AD dataset addresses real-world industrial needs and enables more practical anomaly detection research. Mixed training strategy with synthetic and real defects enhances model performance and applicability to industrial settings.

Abstract: Automatic visual inspection using machine learning plays a key role in achieving zero-defect policies in industry. Research on anomaly detection is constrained by the availability of datasets that capture complex defect appearances and imperfect imaging conditions, which are typical of production processes. Recent benchmarks indicate that most publicly available datasets are biased towards optimal imaging conditions, leading to an overestimation of their applicability in real-world industrial scenarios. To address this gap, we introduce the Industrial Screen Printing Anomaly Detection Dataset (ISP-AD). It presents challenging small and weakly contrasted surface defects embedded within structured patterns exhibiting high permitted design variability. To the best of our knowledge, it is the largest publicly available industrial dataset to date, including both synthetic and real defects collected directly from the factory floor. Beyond benchmarking recent unsupervised anomaly detection methods, experiments on a mixed supervised training strategy, incorporating both synthesized and real defects, were conducted. Experiments show that even a small amount of injected, weakly labeled real defects improves generalization. Furthermore, starting from training on purely synthetic defects, emerging real defective samples can be efficiently integrated into subsequent scalable training. Overall, our findings indicate that model-free synthetic defects can provide a cold-start baseline, whereas a small number of injected real defects refine the decision boundary for previously unseen defect characteristics. The presented unsupervised and supervised dataset splits are designed to emphasize research on unsupervised, self-supervised, and supervised approaches, enhancing their applicability to industrial settings.

Method: The authors propose CCS to measure conceptual consistency of attention heads in CLIP-like models. They conduct soft-pruning experiments to evaluate the importance of high-CCS heads compared to random or low-CCS heads.

Result: High-CCS heads are crucial for model performance - pruning them causes significantly larger performance drops. These heads capture essential concepts and play key roles in out-of-domain detection, concept-specific reasoning, and video-language understanding. However, they also learn spurious correlations that amplify social biases.

Conclusion: CCS is a powerful interpretability metric that exposes the paradox in CLIP models: the same attention heads that drive performance also contribute to social biases, highlighting the need for careful model analysis and bias mitigation strategies.

Abstract: CLIP is one of the most popular foundation models and is heavily used for many vision-language tasks, yet little is known about its inner workings. As CLIP is increasingly deployed in real-world applications, it is becoming even more critical to understand its limitations and embedded social biases to mitigate potentially harmful downstream consequences. However, the question of what internal mechanisms drive both the impressive capabilities as well as problematic shortcomings of CLIP has largely remained unanswered. To bridge this gap, we study the conceptual consistency of text descriptions for attention heads in CLIP-like models. Specifically, we propose Concept Consistency Score (CCS), a novel interpretability metric that measures how consistently individual attention heads in CLIP models align with specific concepts. Our soft-pruning experiments reveal that high CCS heads are critical for preserving model performance, as pruning them leads to a significantly larger performance drop than pruning random or low CCS heads. Notably, we find that high CCS heads capture essential concepts and play a key role in out-of-domain detection, concept-specific reasoning, and video-language understanding. Moreover, we prove that high CCS heads learn spurious correlations which amplify social biases. These results position CCS as a powerful interpretability metric exposing the paradox of performance and social biases in CLIP models.

Method: Proposed CLS-RL for MLLM image classification with verifiable rewards, then introduced No-Thinking-RL (RFT without thinking using simple equality accuracy reward), Think-After-Answer (thinking after answer), and Adaptive-Thinking (MLLMs learn when to think). Evaluated on 6 diverse tasks across different model sizes and types.

Result: 1) Visual perception tasks don’t require thinking during RFT - No-Thinking-RL consistently outperforms/matches Thinking-based RFT. 2) Limited-capability models struggle with high-quality CoT, making Thinking-based RFT less effective. 3) Inconsistencies between thinking and answer tags in thinking-based RFT show lower accuracy. Adaptive-Thinking achieves comparable/better performance than both approaches.

Conclusion: Explicit thinking before verifiable answers may hinder reward convergence and reduce performance. MLLMs can adaptively decide to think or not based on their capabilities and task complexity, with thinking not always being necessary for successful RFT.

Abstract: This paper investigates the role of explicit thinking process in rule-based reinforcement fine-tuning (RFT) for MLLMs. We first propose CLS-RL for MLLM image classification, using verifiable rewards for fine-tuning. Experiments show CLS-RL significantly outperforms SFT and yields a cross-dataset generalization effect. We then rethink and question whether explicit thinking in RFT is always necessary. Challenging the convention that explicit thinking is crucial for the success of RFT, we introduce No-Thinking-RL, exploring RFT without thinking by introducing a simple equality accuracy reward. We evaluate No-Thinking-RL on 6 diverse tasks across different model sizes and types. Experimental results reveal three key findings: 1). Visual perception tasks do not require thinking during RFT, as No-Thinking-RL consistently outperforms or matches Thinking-based RFT across model sizes. 2).} Models with limited capabilities struggle to generate high-quality CoT for RFT, making Thinking-based RFT less effective than No-Thinking-RL. 3). There are inconsistencies between the answers in the thinking and answer tags for some responses of thinking-based RFT, which show lower accuracy than the overall accuracy. We hypothesize that explicit thinking before verifiable answers may hinder reward convergence and reduce performance. To test this hypothesis, we propose Think-After-Answer, which places thinking after the answer to mitigate this effect for experimental verification. Lastly, we conduct a pilot study to explore whether MLLMs can learn when to think during RFT, introducing an Adaptive-Thinking method. Experiments show that it converges to a specific prompt depending on model capability and task complexity, achieving comparable or better performance than both Thinking and No-Thinking-RL. This suggests MLLMs can adaptively decide to think or not based on their capabilities and task complexity.

Method: The researchers collected a dataset of 5,746 images containing both fully visible and partially occluded traffic signs. They compared their custom CNN (96% accuracy) with transfer learning models, with VGG16 achieving the best performance at 99% accuracy when fully unfrozen.

Result: VGG16 with full layer unfreezing achieved 99% accuracy, outperforming the custom CNN. Crucially, models trained only on fully visible signs showed significant performance degradation when tested on occluded signs, demonstrating their lack of robustness to real-world conditions.

Conclusion: Incorporating partially occluded traffic signs in training datasets is essential for developing robust recognition models that can handle complex real-world scenarios. This approach significantly enhances autonomous driving safety by ensuring reliable sign recognition even under challenging visibility conditions.

Abstract: The increasing number of autonomous vehicles and the rapid development of computer vision technologies underscore the particular importance of conducting research on the accuracy of traffic sign recognition. Numerous studies in this field have already achieved significant results, demonstrating high effectiveness in addressing traffic sign recognition tasks. However, the task becomes considerably more complex when a sign is partially obscured by surrounding objects, such as tree branches, billboards, or other elements of the urban environment. In our study, we investigated how partial occlusion of traffic signs affects their recognition. For this purpose, we collected a dataset comprising 5,746 images, including both fully visible and partially occluded signs, and made it publicly available. Using this dataset, we compared the performance of our custom convolutional neural network (CNN), which achieved 96% accuracy, with models trained using transfer learning. The best result was obtained by VGG16 with full layer unfreezing, reaching 99% accuracy. Additional experiments revealed that models trained solely on fully visible signs lose effectiveness when recognizing occluded signs. This highlights the critical importance of incorporating real-world data with partial occlusion into training sets to ensure robust model performance in complex practical scenarios and to enhance the safety of autonomous driving.

Method: Based on InDI iterative method, introduces (i) a regressor network to predict degradation level (mixing asymmetry) and (ii) a degradation-specific normalization module for degradation-aware inference

Result: Solves image splitting and bleedthrough removal tasks, demonstrated applicability on 5 public datasets

Conclusion: indiSplit effectively handles unknown mixing ratios in fluorescence microscopy, outperforming existing methods that assume fixed intensity ratios

Abstract: Fluorescence microscopy, while being a key driver for progress in the life sciences, is also subject to technical limitations. To overcome them, computational multiplexing techniques have recently been proposed, which allow multiple cellular structures to be captured in a single image and later be unmixed. Existing image decomposition methods are trained on a set of superimposed input images and the respective unmixed target images. It is critical to note that the relative strength (mixing ratio) of the superimposed images for a given input is a priori unknown. However, existing methods are trained on a fixed intensity ratio of superimposed inputs, making them not cognizant to the range of relative intensities that can occur in fluorescence microscopy. In this work, we propose a novel method called indiSplit that is cognizant of the severity of the above mentioned mixing ratio. Our idea is based on InDI, a popular iterative method for image restoration, and an ideal starting point to embrace the unknown mixing ratio in any given input. We introduce (i) a suitably trained regressor network that predicts the degradation level (mixing asymmetry) of a given input image and (ii) a degradation-specific normalization module, enabling degradation-aware inference across all mixing ratios. We show that this method solves two relevant tasks in fluorescence microscopy, namely image splitting and bleedthrough removal, and empirically demonstrate the applicability of indiSplit on $5$ public datasets. We will release all sources under a permissive license.

Method: Three AttentionDrop variants: Hard Attention Masking (zeroes out top-k attention logits per query), Blurred Attention Smoothing (applies dynamic Gaussian convolution over attention logits), and Consistency-Regularized AttentionDrop (enforces output stability under multiple perturbations via KL-based consistency loss).

Result: AttentionDrop consistently improves accuracy, calibration, and adversarial robustness compared to standard Dropout, DropConnect, and R-Drop baselines.

Conclusion: AttentionDrop provides effective regularization for Transformers by operating directly on attention mechanisms, addressing overfitting issues in data-limited scenarios.

Abstract: Transformer-based architectures achieve state-of-the-art performance across a wide range of tasks in natural language processing, computer vision, and speech processing. However, their immense capacity often leads to overfitting, especially when training data is limited or noisy. In this research, a unified family of stochastic regularization techniques has been proposed, i.e. AttentionDrop with its three different variants, which operate directly on the self-attention distributions. Hard Attention Masking randomly zeroes out top-k attention logits per query to encourage diverse context utilization, Blurred Attention Smoothing applies a dynamic Gaussian convolution over attention logits to diffuse overly peaked distributions, and Consistency-Regularized AttentionDrop enforces output stability under multiple independent AttentionDrop perturbations via a KL-based consistency loss. Results achieved in the study demonstrate that AttentionDrop consistently improves accuracy, calibration, and adversarial robustness over standard Dropout, DropConnect, and R-Drop baselines

Method: Uses a memory buffer with round-decayed compression for long-context interactions and a decoupled lightweight activation model that integrates easily into existing Video-LLMs for continuous proactive responses. Also creates Stream-IT dataset for streaming video understanding.

Result: Significantly improves streaming understanding capabilities, outperforming proprietary models like GPT-4o and Gemini 1.5 Pro while maintaining competitive performance on standard video benchmarks.

Conclusion: StreamBridge effectively bridges the gap between offline and streaming Video-LLMs, enabling real-time multi-turn interactions and proactive responses without sacrificing standard benchmark performance.

Abstract: We present StreamBridge, a simple yet effective framework that seamlessly transforms offline Video-LLMs into streaming-capable models. It addresses two fundamental challenges in adapting existing models into online scenarios: (1) limited capability for multi-turn real-time understanding, and (2) lack of proactive response mechanisms. Specifically, StreamBridge incorporates (1) a memory buffer combined with a round-decayed compression strategy, supporting long-context multi-turn interactions, and (2) a decoupled, lightweight activation model that can be effortlessly integrated into existing Video-LLMs, enabling continuous proactive responses. To further support StreamBridge, we construct Stream-IT, a large-scale dataset tailored for streaming video understanding, featuring interleaved video-text sequences and diverse instruction formats. Extensive experiments show that StreamBridge significantly improves the streaming understanding capabilities of offline Video-LLMs across various tasks, outperforming even proprietary models such as GPT-4o and Gemini 1.5 Pro. Simultaneously, it achieves competitive or superior performance on standard video understanding benchmarks.

Method: The study deployed NeoMedSys - an integrated platform with AI deployment tools, web-based medical image viewer, annotation system, and hospital information systems. A prospective pragmatic investigation was conducted across two clinical sites (emergency department for TBI and stroke patients) with pre-planned model retraining based on new data.

Result: NeoMedSys enabled iterative improvements in the AI model, significantly enhancing diagnostic accuracy. Sensitivity improved from 79.2% to 90.3%, specificity from 80.7% to 89.3%, and AUC from 0.873 to 0.949. Real-time radiologist feedback facilitated automated bleed detection and segmentation for model retraining.

Conclusion: The platform successfully demonstrated the value of real-time feedback and iterative refinement in clinical AI deployment, with notable performance gains highlighting the effectiveness of the NeoMedSys approach for improving AI model accuracy in radiology applications.

Abstract: Background: There are many challenges and opportunities in the clinical deployment of AI tools in radiology. The current study describes a radiology software platform called NeoMedSys that can enable efficient deployment and refinements of AI models. We evaluated the feasibility and effectiveness of running NeoMedSys for three months in real-world clinical settings and focused on improvement performance of an in-house developed AI model (VIOLA-AI) designed for intracranial hemorrhage (ICH) detection. Methods: NeoMedSys integrates tools for deploying, testing, and optimizing AI models with a web-based medical image viewer, annotation system, and hospital-wide radiology information systems. A prospective pragmatic investigation was deployed using clinical cases of patients presenting to the largest Emergency Department in Norway (site-1) with suspected traumatic brain injury (TBI) or patients with suspected stroke (site-2). We assessed ICH classification performance as VIOLA-AI encountered new data and underwent pre-planned model retraining. Performance metrics included sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve (AUC). Results: NeoMedSys facilitated iterative improvements in the AI model, significantly enhancing its diagnostic accuracy. Automated bleed detection and segmentation were reviewed in near real-time to facilitate re-training VIOLA-AI. The iterative refinement process yielded a marked improvement in classification sensitivity, rising to 90.3% (from 79.2%), and specificity that reached 89.3% (from 80.7%). The bleed detection ROC analysis for the entire sample demonstrated a high area-under-the-curve (AUC) of 0.949 (from 0.873). Model refinement stages were associated with notable gains, highlighting the value of real-time radiologist feedback.

Method: Proposes a framework with ACO for optimal teacher-student model selection and a context-aware predictor that adjusts temperature based on image quality, disease complexity, and teacher confidence. Evaluated on three medical imaging datasets.

Result: Achieved 98.01% accuracy on MRI brain tumor (Kaggle), 92.81% on Figshare MRI, and 96.20% on GastroNet dataset, surpassing previous benchmarks of 97.24%, 91.43%, and 95.00% respectively.

Conclusion: The proposed context-aware framework with ACO optimization significantly outperforms current state-of-the-art methods in medical disease prediction, demonstrating robust knowledge transfer and better handling of medical imaging uncertainties.

Abstract: Medical disease prediction, particularly through imaging, remains a challenging task due to the complexity and variability of medical data, including noise, ambiguity, and differing image quality. Recent deep learning models, including Knowledge Distillation (KD) methods, have shown promising results in brain tumor image identification but still face limitations in handling uncertainty and generalizing across diverse medical conditions. Traditional KD methods often rely on a context-unaware temperature parameter to soften teacher model predictions, which does not adapt effectively to varying uncertainty levels present in medical images. To address this issue, we propose a novel framework that integrates Ant Colony Optimization (ACO) for optimal teacher-student model selection and a novel context-aware predictor approach for temperature scaling. The proposed context-aware framework adjusts the temperature based on factors such as image quality, disease complexity, and teacher model confidence, allowing for more robust knowledge transfer. Additionally, ACO efficiently selects the most appropriate teacher-student model pair from a set of pre-trained models, outperforming current optimization methods by exploring a broader solution space and better handling complex, non-linear relationships within the data. The proposed framework is evaluated using three publicly available benchmark datasets, each corresponding to a distinct medical imaging task. The results demonstrate that the proposed framework significantly outperforms current state-of-the-art methods, achieving top accuracy rates: 98.01% on the MRI brain tumor (Kaggle) dataset, 92.81% on the Figshare MRI dataset, and 96.20% on the GastroNet dataset. This enhanced performance is further evidenced by the improved results, surpassing existing benchmarks of 97.24% (Kaggle), 91.43% (Figshare), and 95.00% (GastroNet).

Method: Created TT-DF dataset with 6,120 forged videos using multiple human image animation models and generative configurations. Proposed TOF-Net that exploits spatiotemporal inconsistencies and optical flow distribution differences between natural and forged data.

Result: TOF-Net achieves favorable performance on TT-DF, outperforming current state-of-the-art extendable facial forgery detection models.

Conclusion: The TT-DF dataset and TOF-Net model address the gap in human body forgery detection, providing comprehensive resources and effective detection methods for this emerging security concern.

Abstract: The emergence and popularity of facial deepfake methods spur the vigorous development of deepfake datasets and facial forgery detection, which to some extent alleviates the security concerns about facial-related artificial intelligence technologies. However, when it comes to human body forgery, there has been a persistent lack of datasets and detection methods, due to the later inception and complexity of human body generation methods. To mitigate this issue, we introduce TikTok-DeepFake (TT-DF), a novel large-scale diffusion-based dataset containing 6,120 forged videos with 1,378,857 synthetic frames, specifically tailored for body forgery detection. TT-DF offers a wide variety of forgery methods, involving multiple advanced human image animation models utilized for manipulation, two generative configurations based on the disentanglement of identity and pose information, as well as different compressed versions. The aim is to simulate any potential unseen forged data in the wild as comprehensively as possible, and we also furnish a benchmark on TT-DF. Additionally, we propose an adapted body forgery detection model, Temporal Optical Flow Network (TOF-Net), which exploits the spatiotemporal inconsistencies and optical flow distribution differences between natural data and forged data. Our experiments demonstrate that TOF-Net achieves favorable performance on TT-DF, outperforming current state-of-the-art extendable facial forgery detection models. For our TT-DF dataset, please refer to https://github.com/HashTAG00002/TT-DF.

Method: Proposes MFDCD framework with two key components: Dynamic Frequency Coupler (DFC) using wavelet transform to model linear transitions, and Textual Frequency Filter (TFF) encoding semantic priors into frequency-domain graphs with filter banks for semantic alignment.

Result: MFDCD achieves state-of-the-art performance on the new RB-SCD dataset and three public change detection datasets, demonstrating superior capability in road and bridge change detection.

Conclusion: The proposed frequency-domain multimodal approach effectively addresses the unique challenges of road and bridge change detection, with the RB-SCD dataset serving as the first benchmark for systematic semantic change detection of transportation infrastructure.

Abstract: Accurate detection of road and bridge changes is crucial for urban planning and transportation management, yet presents unique challenges for general change detection (CD). Key difficulties arise from maintaining the continuity of roads and bridges as linear structures and disambiguating visually similar land covers (e.g., road construction vs. bare land). Existing spatial-domain models struggle with these issues, further hindered by the lack of specialized, semantically rich datasets. To fill these gaps, we introduce the Road and Bridge Semantic Change Detection (RB-SCD) dataset. As the first benchmark to systematically target semantic change detection of roads and bridges, RB-SCD offers comprehensive fine-grained annotations for 11 semantic change categories. This enables a detailed analysis of traffic infrastructure evolution. Building on this, we propose a novel framework, the Multimodal Frequency-Driven Change Detector (MFDCD). MFDCD integrates multimodal features in the frequency domain through two key components: (1) the Dynamic Frequency Coupler (DFC), which leverages wavelet transform to decompose visual features, enabling it to robustly model the continuity of linear transitions; and (2) the Textual Frequency Filter (TFF), which encodes semantic priors into frequency-domain graphs and applies filter banks to align them with visual features, resolving semantic ambiguities. Experiments demonstrate the state-of-the-art performance of MFDCD on RB-SCD and three public CD datasets. The code will be available at https://github.com/DaGuangDaGuang/RB-SCD.

Method: The GRE Suite framework includes: GRE30K dataset for fine-grained analysis, a multi-stage reasoning model that progressively infers scene attributes and semantic features, and GREval-Bench evaluation framework for comprehensive assessment.

Result: Experimental results show GRE significantly outperforms existing methods across all granularities of geo-localization tasks, demonstrating the efficacy of reasoning-augmented VLMs.

Conclusion: The GRE Suite framework successfully addresses limitations in geo-localization by providing structured reasoning chains, leading to more accurate and interpretable location inference.

Abstract: Recent advances in Visual Language Models (VLMs) have demonstrated exceptional performance in visual reasoning tasks. However, geo-localization presents unique challenges, requiring the extraction of multigranular visual cues from images and their integration with external world knowledge for systematic reasoning. Current approaches to geo-localization tasks often lack robust reasoning mechanisms and explainability, limiting their effectiveness. To address these limitations, we propose the Geo Reason Enhancement (GRE) Suite, a novel framework that augments VLMs with structured reasoning chains for accurate and interpretable location inference. The GRE Suite is systematically developed across three key dimensions: dataset, model, and benchmark. First, we introduce GRE30K, a high-quality geo-localization reasoning dataset designed to facilitate fine-grained visual and contextual analysis. Next, we present the GRE model, which employs a multi-stage reasoning strategy to progressively infer scene attributes, local details, and semantic features, thereby narrowing down potential geographic regions with enhanced precision. Finally, we construct the Geo Reason Evaluation Benchmark (GREval-Bench), a comprehensive evaluation framework that assesses VLMs across diverse urban, natural, and landmark scenes to measure both coarse-grained (e.g., country, continent) and fine-grained (e.g., city, street) localization performance. Experimental results demonstrate that GRE significantly outperforms existing methods across all granularities of geo-localization tasks, underscoring the efficacy of reasoning-augmented VLMs in complex geographic inference. Code and data will be released at https://github.com/Thorin215/GRE.

Method: Two-stage pipeline: 1) Supervised fine-tuning on large-scale procedurally generated data, 2) Reinforcement learning fine-tuning using online feedback with Group Relative Preference Optimization (GRPO). Leverages vision-language models and explores RL fine-tuning for CAD tasks.

Result: Outperforms existing single-modal approaches on all three input modalities in DeepCAD benchmark. After RL fine-tuning, achieves new state-of-the-art on three challenging datasets including real-world data.

Conclusion: Multi-modal CAD reconstruction with RL fine-tuning significantly improves performance over single-modal approaches, demonstrating the effectiveness of the proposed training paradigm for CAD applications.

Abstract: Computer-Aided Design (CAD) plays a central role in engineering and manufacturing, making it possible to create precise and editable 3D models. Using a variety of sensor or user-provided data as inputs for CAD reconstruction can democratize access to design applications. However, existing methods typically focus on a single input modality, such as point clouds, images, or text, which limits their generalizability and robustness. Leveraging recent advances in vision-language models (VLM), we propose a multi-modal CAD reconstruction model that simultaneously processes all three input modalities. Inspired by large language model (LLM) training paradigms, we adopt a two-stage pipeline: supervised fine-tuning (SFT) on large-scale procedurally generated data, followed by reinforcement learning (RL) fine-tuning using online feedback, obtained programatically. Furthermore, we are the first to explore RL fine-tuning of LLMs for CAD tasks demonstrating that online RL algorithms such as Group Relative Preference Optimization (GRPO) outperform offline alternatives. In the DeepCAD benchmark, our SFT model outperforms existing single-modal approaches in all three input modalities simultaneously. More importantly, after RL fine-tuning, cadrille sets new state-of-the-art on three challenging datasets, including a real-world one.

Method: OSPO consists of four components: (1) Initial Prompt Generation, (2) Hard Preference Pair Generation, (3) Filtering and Selection, and (4) Object-centric Preference Optimization with Conditional Preference Loss, which explicitly constructs and leverages object-level hard negative data for optimization.

Result: Extensive experiments on compositional image generation benchmarks show that OSPO significantly improves fine-grained alignment in text-to-image generation, outperforming both prior self-improving methods and diffusion-based specialized image generation models.

Conclusion: OSPO effectively addresses the object hallucination problem in text-to-image generation through its object-centric self-improving framework, demonstrating superior performance in achieving fine-grained text-image alignment without relying on external large-scale data or models.

Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have enabled models to perform both understanding and generation of multimodal data in a unified manner. However, achieving a fine-grained alignment between input prompts and generated images remains a major challenge especially in text-to-image generation. Therefore, recent works have introduced self-improving mechanisms based on self-generated data and self-feedback to efficiently mitigate this challenge without relying on external large-scale data or models. However, existing self-improving approaches have not focused on fine-grained visual details especially at the object level in generating training data or providing a feedback, and thus they still struggle to resolve the object hallucination problem in text-to-image generation. To tackle this problem, we propose an Object-centric Self-improving Preference Optimization (OSPO), a self-improving framework for enhancing object-level text-image alignment. OSPO is designed to explicitly address the need for constructing and leveraging object-level hard negative data and an object-centric optimization in improving object-specific fidelity. In specific, OSPO consists of: (1) Initial Prompt Generation (2) Hard Preference Pair Generation (3) Filtering and Selection (4) Object-centric Preference Optimization with Conditional Preference Loss. Extensive experiments on compositional image generation benchmarks demonstrate that OSPO significantly improves fine-grained alignment in text-to-image generation, surpassing not only prior self-improving methods but also diffusion-based specialized image generation models.

Method: SpatialMind decomposes complex scenes/questions into interpretable reasoning steps; ScanForgeQA is a scalable QA dataset built from 3D simulation scenes through automated construction for fine-tuning.

Result: Extensive experiments across multiple benchmarks demonstrate individual and combined effectiveness of prompting and fine-tuning strategies.

Conclusion: The framework successfully enhances 3D spatial reasoning in VLMs and provides insights for future visual-spatial understanding research.

Abstract: Visual-spatial understanding, the ability to infer object relationships and layouts from visual input, is fundamental to downstream tasks such as robotic navigation and embodied interaction. However, existing methods face spatial uncertainty and data scarcity, limiting the 3D spatial reasoning capability of pre-trained vision-language models (VLMs). To address these challenges, we present a unified framework for enhancing 3D spatial reasoning in pre-trained VLMs without modifying their architecture. This framework combines SpatialMind, a structured prompting strategy that decomposes complex scenes and questions into interpretable reasoning steps, with ScanForgeQA, a scalable question-answering dataset built from diverse 3D simulation scenes through an automated construction process designed for fine-tuning. Extensive experiments across multiple benchmarks demonstrate the individual and combined effectiveness of our prompting and fine-tuning strategies, and yield insights that may inspire future research on visual-spatial understanding.

Method: Used three CLIP-based VLMs and performed controlled self-attention ablations on a carefully designed probing task to analyze the interaction between visual and language components.

Result: CLIP visual representations provide ready-to-read semantic information to language decoders, but language decoders can largely compensate for deficiencies in visual representations and recover performance when visual contextualization is reduced.

Conclusion: There is a dynamic division of labor in VLMs, motivating future architectures that can offload more visual processing to language decoders.

Abstract: Many vision-language models (VLMs) that prove very effective at a range of multimodal task, build on CLIP-based vision encoders, which are known to have various limitations. We investigate the hypothesis that the strong language backbone in VLMs compensates for possibly weak visual features by contextualizing or enriching them. Using three CLIP-based VLMs, we perform controlled self-attention ablations on a carefully designed probing task. Our findings show that despite known limitations, CLIP visual representations offer ready-to-read semantic information to the language decoder. However, in scenarios of reduced contextualization in the visual representations, the language decoder can largely compensate for the deficiency and recover performance. This suggests a dynamic division of labor in VLMs and motivates future architectures that offload more visual processing to the language decoder.

Method: Created 3D-SynthPlace dataset via ‘GPT synthesize, Human inspect’ pipeline (17,000 scenes across 4 room types). Developed OptiScene LLM with two-stage training: Stage I - supervised fine-tuning for spatial descriptions and object placements; Stage II - multi-turn direct preference optimization to align with human preferences.

Result: Extensive experiments show OptiScene outperforms traditional prompt-driven and learning-based baselines. It also demonstrates promising potential in interactive tasks like scene editing and robot navigation.

Conclusion: The proposed 3D-SynthPlace dataset and OptiScene LLM provide an effective solution for indoor layout generation, addressing limitations of existing methods and showing strong performance and generalization capabilities.

Abstract: Automatic indoor layout generation has attracted increasing attention due to its potential in interior design, virtual environment construction, and embodied AI. Existing methods fall into two categories: prompt-driven approaches that leverage proprietary LLM services (e.g., GPT APIs) and learning-based methods trained on layout data upon diffusion-based models. Prompt-driven methods often suffer from spatial inconsistency and high computational costs, while learning-based methods are typically constrained by coarse relational graphs and limited datasets, restricting their generalization to diverse room categories. In this paper, we revisit LLM-based indoor layout generation and present 3D-SynthPlace, a large-scale dataset that combines synthetic layouts generated via a ‘GPT synthesize, Human inspect’ pipeline, upgraded from the 3D-Front dataset. 3D-SynthPlace contains nearly 17,000 scenes, covering four common room types – bedroom, living room, kitchen, and bathroom – enriched with diverse objects and high-level spatial annotations. We further introduce OptiScene, a strong open-source LLM optimized for indoor layout generation, fine-tuned based on our 3D-SynthPlace dataset through our two-stage training. For the warum-up stage I, we adopt supervised fine-tuning (SFT), which is taught to first generate high-level spatial descriptions then conditionally predict concrete object placements. For the reinforcing stage II, to better align the generated layouts with human design preferences, we apply multi-turn direct preference optimization (DPO), which significantly improving layout quality and generation success rates. Extensive experiments demonstrate that OptiScene outperforms traditional prompt-driven and learning-based baselines. Moreover, OptiScene shows promising potential in interactive tasks such as scene editing and robot navigation.

Method: Proposes DualEdit which edits both textual and visual modalities at their peak sensitivity layers, with a gating module in the more sensitive textual modality to balance knowledge update and preservation.

Result: DualEdit demonstrates superiority over state-of-the-art VLM editing baselines and adapted LLM editing methods across multiple VLM backbones and benchmark datasets.

Conclusion: Editing both modalities efficiently updates knowledge but requires careful layer selection and gating mechanisms to avoid compromising the model’s original capabilities.

Abstract: Model editing aims to efficiently update a pre-trained model’s knowledge without the need for time-consuming full retraining. While existing pioneering editing methods achieve promising results, they primarily focus on editing single-modal language models (LLMs). However, for vision-language models (VLMs), which involve multiple modalities, the role and impact of each modality on editing performance remain largely unexplored. To address this gap, we explore the impact of textual and visual modalities on model editing and find that: (1) textual and visual representations reach peak sensitivity at different layers, reflecting their varying importance; and (2) editing both modalities can efficiently update knowledge, but this comes at the cost of compromising the model’s original capabilities. Based on our findings, we propose DualEdit, an editor that modifies both textual and visual modalities at their respective key layers. Additionally, we introduce a gating module within the more sensitive textual modality, allowing DualEdit to efficiently update new knowledge while preserving the model’s original information. We evaluate DualEdit across multiple VLM backbones and benchmark datasets, demonstrating its superiority over state-of-the-art VLM editing baselines as well as adapted LLM editing methods on different evaluation metrics. Codes are available at https://github.com/zhiyiscs/DualEdit

Method: The researchers trained both a custom CNN and EfficientNetB0 on an extended dataset of 126 original photographs that were augmented to generate additional images. Performance was evaluated using accuracy, precision, recall, F1 score, and mean average precision (mAP).

Result: The custom CNN achieved 92.8% accuracy while EfficientNetB0 achieved 98.4% accuracy. Analysis of confusion matrices revealed the strengths and weaknesses of each model.

Conclusion: Using pre-trained models like EfficientNetB0 significantly improves classification accuracy and generalization for bazaar tent classification, confirming the effectiveness of transfer learning in this domain.

Abstract: This research paper proposes an improved deep learning model for classifying tents in street bazaars, comparing a custom Convolutional Neural Network (CNN) with EfficientNetB0. This is a critical task for market organization with a tent classification, but manual methods in the past have been inefficient. Street bazaars represent a vital economic hub in many regions, yet their unstructured nature poses significant challenges for the automated classification of market infrastructure, such as tents. In Kyrgyzstan, more than a quarter of the country’s GDP is derived from bazaars. While CNNs have been widely applied to object recognition, their application to bazaar-specific tasks remains underexplored. Here, we build upon our original approach by training on an extended set of 126 original photographs that were augmented to generate additional images. This dataset is publicly available for download on Kaggle. A variety of performance metrics, such as accuracy, precision, recall, F1 score, and mean average precision (mAP), were used to assess the models comparatively, providing a more extensive analysis of classification performance. The results show that the CNN custom model achieved 92.8% accuracy, and EfficientNetB0 showed 98.4% accuracy results, confirming the effectiveness of transfer learning in the bazaar image classification. Also, when analyzing the confusion matrix, the analysis reveals the weaknesses and strengths of each model. These findings suggest that using a pre-trained model such as EfficientNetB0 significantly improves classification accuracy and generalization.

Method: A reinforcement learning-based post-training framework that enhances MLLMs’ visual recognition and personalized generation capabilities without requiring large-scale caption datasets.

Result: The RL-based approach significantly outperforms existing SFT-based baselines, particularly in challenging multi-concept image captioning tasks.

Conclusion: RL-based post-training is an effective alternative to SFT for improving MLLM personalization, addressing data scarcity issues while enhancing both visual recognition and personalized caption generation.

Abstract: Recent multi-modal large language models (MLLMs) often struggle to generate personalized image captions, even when trained on high-quality captions. In this work, we observe that such limitations persist in existing post-training-based MLLM personalization methods. Specifically, despite being post-tuned with large-scale caption data through supervised fine-tuning (SFT), these models frequently fail to produce faithful descriptions in real-world scenarios, such as multi-concept image captioning. However, acquiring large-scale, high-quality captions for such complex settings is both costly and difficult. To address the data-centric nature of SFT, we propose a reinforcement learning (RL)-based post-training framework. To the best of our knowledge, this is the first RL-based approach to post-train MLLMs for personalized image captioning. Our method significantly enhances both visual recognition and personalized generation capabilities of MLLMs, and consistently outperforms existing SFT-based baselines, especially in the challenging multi-concept image captioning task.

Method: ViLU constructs uncertainty-aware multi-modal representations by integrating visual embeddings, predicted textual embeddings, and image-conditioned textual representations via cross-attention. It trains an uncertainty predictor as a binary classifier using weighted binary cross-entropy loss.

Result: Extensive experiments show significant gains compared to state-of-the-art failure prediction methods on datasets including ImageNet-1k, CC12M, and LAION-400M. Ablation studies confirm the importance of the proposed architecture and training approach.

Conclusion: ViLU provides an effective post-hoc uncertainty quantification framework for VLMs that is loss-agnostic and works well when only vision and text embeddings are available without direct model access.

Abstract: Reliable Uncertainty Quantification (UQ) and failure prediction remain open challenges for Vision-Language Models (VLMs). We introduce ViLU, a new Vision-Language Uncertainty quantification framework that contextualizes uncertainty estimates by leveraging all task-relevant textual representations. ViLU constructs an uncertainty-aware multi-modal representation by integrating the visual embedding, the predicted textual embedding, and an image-conditioned textual representation via cross-attention. Unlike traditional UQ methods based on loss prediction, ViLU trains an uncertainty predictor as a binary classifier to distinguish correct from incorrect predictions using a weighted binary cross-entropy loss, making it loss-agnostic. In particular, our proposed approach is well-suited for post-hoc settings, where only vision and text embeddings are available without direct access to the model itself. Extensive experiments on diverse datasets show the significant gains of our method compared to state-of-the-art failure prediction methods. We apply our method to standard classification datasets, such as ImageNet-1k, as well as large-scale image-caption datasets like CC12M and LAION-400M. Ablation studies highlight the critical role of our architecture and training in achieving effective uncertainty quantification. Our code is publicly available and can be found here: https://github.com/ykrmm/ViLU.

Method: Proposes Deformable Dynamic Convolutional Network (DDCN) with two key components: deformable layers with learnable offsets for flexible receptive fields, and dynamic layers that generate region-specific filters to adapt to spatio-temporal variations.

Result: Extensive experiments on four real-world traffic datasets show DDCN achieves competitive predictive performance while significantly reducing computational costs compared to existing methods.

Conclusion: DDCN effectively captures non-Euclidean spatial structures and spatio-temporal heterogeneity, demonstrating strong potential for large-scale and real-time traffic prediction deployment.

Abstract: Traffic prediction is a critical component of intelligent transportation systems, enabling applications such as congestion mitigation and accident risk prediction. While recent research has explored both graph-based and grid-based approaches, key limitations remain. Graph-based methods effectively capture non-Euclidean spatial structures but often incur high computational overhead, limiting their practicality in large-scale systems. In contrast, grid-based methods, which primarily leverage Convolutional Neural Networks (CNNs), offer greater computational efficiency but struggle to model irregular spatial patterns due to the fixed shape of their filters. Moreover, both approaches often fail to account for inherent spatio-temporal heterogeneity, as they typically apply a shared set of parameters across diverse regions and time periods. To address these challenges, we propose the Deformable Dynamic Convolutional Network (DDCN), a novel CNN-based architecture that integrates both deformable and dynamic convolution operations. The deformable layer introduces learnable offsets to create flexible receptive fields that better align with spatial irregularities, while the dynamic layer generates region-specific filters, allowing the model to adapt to varying spatio-temporal traffic patterns. By combining these two components, DDCN effectively captures both non-Euclidean spatial structures and spatio-temporal heterogeneity. Extensive experiments on four real-world traffic datasets demonstrate that DDCN achieves competitive predictive performance while significantly reducing computational costs, underscoring its potential for large-scale and real-time deployment.

Method: CR-Net integrates Structure-induced Hierarchical Feature Adaptation (SHFA) for structure-aware feature adaptation and Reliable Structural Adjacency Alignment (RSAA) for reliable semantic alignment using secure adjacency sets.

Result: Experimental results show CR-Net significantly enhances cross-resolution adaptation by preserving intra-domain structures and improving discriminability, achieving state-of-the-art performance.

Conclusion: The proposed CR-Net effectively addresses cross-resolution SAR target detection challenges through structure-aware adaptation and reliable semantic alignment, demonstrating superior performance over existing methods.

Abstract: In recent years, continuous improvements in SAR resolution have significantly benefited applications such as urban monitoring and target detection. However, the improvement in resolution leads to increased discrepancies in scattering characteristics, posing challenges to the generalization ability of target detection models. While domain adaptation technology is a potential solution, the inevitable discrepancies caused by resolution differences often lead to blind feature adaptation and unreliable semantic propagation, ultimately degrading the domain adaptation performance. To address these challenges, this paper proposes a novel SAR target detection method (termed CR-Net), that incorporates structure priors and evidential learning theory into the detection model, enabling reliable domain adaptation for cross-resolution detection. To be specific, CR-Net integrates Structure-induced Hierarchical Feature Adaptation (SHFA) and Reliable Structural Adjacency Alignment (RSAA). SHFA module is introduced to establish structural correlations between targets and achieve structure-aware feature adaptation, thereby enhancing the interpretability of the feature adaptation process. Afterwards, the RSAA module is proposed to enhance reliable semantic alignment, by leveraging the secure adjacency set to transfer valuable discriminative knowledge from the source domain to the target domain. This further improves the discriminability of the detection model in the target domain. Based on experimental results from different-resolution datasets,the proposed CR-Net significantly enhances cross-resolution adaptation by preserving intra-domain structures and improving discriminability. It achieves state-of-the-art (SOTA) performance in cross-resolution SAR target detection.

Method: Integrates multiscale visual-textual alignment metrics (localized patch affinity, global semantic coherence, contextual attention patterns) with entropy-sensitive mechanism to dynamically balance watermark strength and semantic preservation without model retraining.

Result: Achieves 7.4% lower PPL and 26.6% higher BLEU than conventional methods, with 98.8% AUC detection accuracy and 96.1% attack resilience against paraphrasing and synonym substitution.

Conclusion: Establishes new standards for quality-preserving multimodal watermarking by maintaining text-visual consistency while providing robust intellectual property protection.

Abstract: Vision-language models demand watermarking solutions that protect intellectual property without compromising multimodal coherence. Existing text watermarking methods disrupt visual-textual alignment through biased token selection and static strategies, leaving semantic-critical concepts vulnerable. We propose VLA-Mark, a vision-aligned framework that embeds detectable watermarks while preserving semantic fidelity through cross-modal coordination. Our approach integrates multiscale visual-textual alignment metrics, combining localized patch affinity, global semantic coherence, and contextual attention patterns, to guide watermark injection without model retraining. An entropy-sensitive mechanism dynamically balances watermark strength and semantic preservation, prioritizing visual grounding during low-uncertainty generation phases. Experiments show 7.4% lower PPL and 26.6% higher BLEU than conventional methods, with near-perfect detection (98.8% AUC). The framework demonstrates 96.1% attack resilience against attacks such as paraphrasing and synonym substitution, while maintaining text-visual consistency, establishing new standards for quality-preserving multimodal watermarking

Method: MCGA learns transferable spectral priors via mixture-of-codebooks from heterogeneous HSI datasets, then aligns RGB features with these priors through grayscale-aware photometric attention. It uses top-K attention design and test-time adaptation for efficiency and robustness.

Result: Experiments show state-of-the-art accuracy, strong cross-dataset generalization, and 4-5x faster inference compared to existing methods on benchmarks and real-world data.

Conclusion: MCGA provides an efficient and robust solution for hyperspectral image reconstruction that explicitly addresses the ill-posed problem through spectral priors and photometric consistency, achieving superior performance with faster inference.

Abstract: Reconstructing hyperspectral images (HSIs) from RGB inputs provides a cost-effective alternative to hyperspectral cameras, but reconstructing high-dimensional spectra from three channels is inherently ill-posed. Existing methods typically directly regress RGB-to-HSI mappings using large attention networks, which are computationally expensive and handle ill-posedness only implicitly. We propose MCGA, a Mixture-of-Codebooks with Grayscale-aware Attention framework that explicitly addresses these challenges using spectral priors and photometric consistency. MCGA first learns transferable spectral priors via a mixture-of-codebooks (MoC) from heterogeneous HSI datasets, then aligns RGB features with these priors through grayscale-aware photometric attention (GANet). Efficiency and robustness are further improved via top-K attention design and test-time adaptation (TTA). Experiments on benchmarks and real-world data demonstrate the state-of-the-art accuracy, strong cross-dataset generalization, and 4-5x faster inference. Codes will be available once acceptance at https://github.com/Fibonaccirabbit/MCGA.

Method: The authors created the RSCC dataset containing 62,315 pre-/post-disaster image pairs covering various disasters (earthquakes, floods, wildfires, etc.) paired with human-like change captions.

Result: RSCC enables robust training and evaluation of vision-language models for disaster-aware bi-temporal understanding, facilitating detailed disaster-related analysis.

Conclusion: The RSCC dataset bridges the temporal and semantic divide in remote sensing data, paving the way for more accurate, interpretable, and scalable vision-language applications in disaster monitoring.

Abstract: Remote sensing is critical for disaster monitoring, yet existing datasets lack temporal image pairs and detailed textual annotations. While single-snapshot imagery dominates current resources, it fails to capture dynamic disaster impacts over time. To address this gap, we introduce the Remote Sensing Change Caption (RSCC) dataset, a large-scale benchmark comprising 62,315 pre-/post-disaster image pairs (spanning earthquakes, floods, wildfires, and more) paired with rich, human-like change captions. By bridging the temporal and semantic divide in remote sensing data, RSCC enables robust training and evaluation of vision-language models for disaster-aware bi-temporal understanding. Our results highlight RSCC’s ability to facilitate detailed disaster-related analysis, paving the way for more accurate, interpretable, and scalable vision-language applications in remote sensing. Code and dataset are available at https://github.com/Bili-Sakura/RSCC.

Method: Introduces a region-of-interest (ROI) processor that adaptively integrates fine-grained regional features with global context, using progressive training strategy for hierarchical multimodal alignment. Built on MedRegion-500k, a comprehensive medical image-text corpus with extensive regional annotations.

Result: Extensive experiments on image-text retrieval, zero-shot classification, and visual question answering show RegionMed-CLIP consistently exceeds state-of-the-art vision language models by a wide margin.

Conclusion: Region-aware contrastive pre-training is critical for advancing multimodal medical image understanding, positioning RegionMed-CLIP as a robust foundation for the field.

Abstract: Medical image understanding plays a crucial role in enabling automated diagnosis and data-driven clinical decision support. However, its progress is impeded by two primary challenges: the limited availability of high-quality annotated medical data and an overreliance on global image features, which often miss subtle but clinically significant pathological regions. To address these issues, we introduce RegionMed-CLIP, a region-aware multimodal contrastive learning framework that explicitly incorporates localized pathological signals along with holistic semantic representations. The core of our method is an innovative region-of-interest (ROI) processor that adaptively integrates fine-grained regional features with the global context, supported by a progressive training strategy that enhances hierarchical multimodal alignment. To enable large-scale region-level representation learning, we construct MedRegion-500k, a comprehensive medical image-text corpus that features extensive regional annotations and multilevel clinical descriptions. Extensive experiments on image-text retrieval, zero-shot classification, and visual question answering tasks demonstrate that RegionMed-CLIP consistently exceeds state-of-the-art vision language models by a wide margin. Our results highlight the critical importance of region-aware contrastive pre-training and position RegionMed-CLIP as a robust foundation for advancing multimodal medical image understanding.

Method: CLIPTTA uses gradient-based adaptation with a soft contrastive loss that aligns with CLIP’s pre-training objective. The method includes theoretical analysis of gradients and batch-aware design to prevent collapse. It’s extended to open-set scenarios with Outlier Contrastive Exposure (OCE) loss for improved OOD detection.

Result: Evaluated on 75 datasets with diverse distribution shifts, CLIPTTA consistently outperforms entropy-based objectives and is competitive with state-of-the-art TTA methods, showing more stable performance across different shifts.

Conclusion: CLIPTTA provides an effective test-time adaptation approach for VLMs that is better aligned with their contrastive pre-training, mitigating common failure modes and improving generalization under distribution shifts.

Abstract: Vision-language models (VLMs) like CLIP exhibit strong zero-shot capabilities but often fail to generalize under distribution shifts. Test-time adaptation (TTA) allows models to update at inference time without labeled data, typically via entropy minimization. However, this objective is fundamentally misaligned with the contrastive image-text training of VLMs, limiting adaptation performance and introducing failure modes such as pseudo-label drift and class collapse. We propose CLIPTTA, a new gradient-based TTA method for vision-language models that leverages a soft contrastive loss aligned with CLIP’s pre-training objective. We provide a theoretical analysis of CLIPTTA’s gradients, showing how its batch-aware design mitigates the risk of collapse. We further extend CLIPTTA to the open-set setting, where both in-distribution (ID) and out-of-distribution (OOD) samples are encountered, using an Outlier Contrastive Exposure (OCE) loss to improve OOD detection. Evaluated on 75 datasets spanning diverse distribution shifts, CLIPTTA consistently outperforms entropy-based objectives and is highly competitive with state-of-the-art TTA methods, outperforming them on a large number of datasets and exhibiting more stable performance across diverse shifts.

Method: Constructed SAR-TEXT dataset using SAR-Narrator framework (multi-stage strategy for generating textual descriptions), then validated on three tasks: image-text retrieval, image captioning, and VQA using SAR-RS-CLIP, SAR-RS-CoCa, and SAR-GPT models.

Result: SAR-RS-CLIP improved average recall by 12.97% and 10.0% on OSdataset_512 and HRSID; SAR-RS-CoCa outperformed original CoCa in BLEU-4, SPICE, CIDEr; SAR-GPT beat baselines on multiple SAR-VQA datasets with stronger semantic understanding.

Conclusion: SAR-TEXT dataset enables significant improvements in SAR image understanding tasks, and SAR-Narrator provides a flexible tool for community to construct larger-scale SAR datasets.

Abstract: Vision Language Models (VLMs) have achieved remarkable breakthroughs in the field of remote sensing in recent years. Synthetic Aperture Radar (SAR) imagery, with its all-weather capability, is essential in remote sensing, yet the lack of large-scale, high-quality SAR image-text datasets hinders its semantic understanding. In this paper, we construct SAR-TEXT, a large-scale and high-quality dataset consisting of over 130,000 SAR image-text pairs. To construct the SAR-TEXT dataset, we design the SAR-Narrator framework, which generates textual descriptions for SAR images through a multi-stage strategy. To verify the effectiveness of the SAR-TEXT dataset, we conduct experiments on three typical vision-language tasks: image-text retrieval, image captioning, and visual question answering (VQA). Specifically, we construct three representative models on SAR-TEXT: SAR-RS-CLIP, SAR-RS-CoCa, and SAR-GPT. SAR-RS-CLIP achieves notable improvements in retrieval performance, boosting average recall by 12.97% and 10.0% on the OSdataset_512 and HRSID test sets, respectively. In the captioning task, SAR-RS-CoCa achieves significant improvements over the original CoCa models in terms of BLEU-4, SPICE, and CIDEr scores. In the VQA task, SAR-GPT outperforms baseline and single-stage models on multiple SAR-VQA datasets, demonstrating stronger semantic understanding and reasoning ability, as further confirmed by qualitative results. It is worth noting that, as a flexible captioning tool, SAR-Narrator can be readily adopted by the community to construct larger-scale SAR image-text datasets. All code, pretrained models, and the SAR-Text dataset are publicly available at: https://github.com/YiguoHe/SAR-TEXT.

Method: The framework partitions images into texture-homogeneous regions using entropy-driven algorithm, parameterizes each region as a 2D Gaussian (position and shape) with texture features, trains a transformer to optimize Gaussian parameters, and uses differentiable splatting-based renderer for decoding.

Result: GPSToken achieves state-of-the-art performance with rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks using only 128 tokens.

Conclusion: GPSToken enables efficient two-stage generation (structural layout synthesis followed by texture generation) and demonstrates superior performance in adaptive image tokenization compared to conventional grid-based methods.

Abstract: Effective and efficient tokenization plays an important role in image representation and generation. Conventional methods, constrained by uniform 2D/1D grid tokenization, are inflexible to represent regions with varying shapes and textures and at different locations, limiting their efficacy of feature representation. In this work, we propose $\textbf{GPSToken}$, a novel $\textbf{G}$aussian $\textbf{P}$arameterized $\textbf{S}$patially-adaptive $\textbf{Token}$ization framework, to achieve non-uniform image tokenization by leveraging parametric 2D Gaussians to dynamically model the shape, position, and textures of different image regions. We first employ an entropy-driven algorithm to partition the image into texture-homogeneous regions of variable sizes. Then, we parameterize each region as a 2D Gaussian (mean for position, covariance for shape) coupled with texture features. A specialized transformer is trained to optimize the Gaussian parameters, enabling continuous adaptation of position/shape and content-aware feature extraction. During decoding, Gaussian parameterized tokens are reconstructed into 2D feature maps through a differentiable splatting-based renderer, bridging our adaptive tokenization with standard decoders for end-to-end training. GPSToken disentangles spatial layout (Gaussian parameters) from texture features to enable efficient two-stage generation: structural layout synthesis using lightweight networks, followed by structure-conditioned texture generation. Experiments demonstrate the state-of-the-art performance of GPSToken, which achieves rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks using 128 tokens, respectively. Codes and models of GPSToken can be found at $\href{https://github.com/xtudbxk/GPSToken}{https://github.com/xtudbxk/GPSToken}$.

Method: Formulates RGB-to-HSI as a physics-grounded inverse problem with nuclear norm regularization in a learnable transform domain. Introduces data-adaptive low-rank subspace SVT operator to avoid full SVD costs. USCTNet couples parameter estimation with learnable proximal updates through deep unfolding.

Result: Extensive experiments on standard benchmarks show consistent improvements over state-of-the-art RGB-based methods in reconstruction accuracy.

Conclusion: The proposed USCTNet framework effectively addresses physical inconsistency in HSI reconstruction by explicitly estimating CSS and illumination, while the learnable low-rank subspace approach provides stable and accurate results.

Abstract: Reconstructing hyperspectral images (HSIs) from a single RGB image is ill-posed and can become physically inconsistent when the camera spectral sensitivity (CSS) and scene illumination are misspecified. We formulate RGB-to-HSI reconstruction as a physics-grounded inverse problem regularized by a nuclear norm in a learnable transform domain, and we explicitly estimate CSS and illumination to define the forward operator embedded in each iteration, ensuring colorimetric consistency. To avoid the cost and instability of full singular-value decompositions (SVDs) required by singular-value thresholding (SVT), we introduce a data-adaptive low-rank subspace SVT operator. Building on these components, we develop USCTNet, a deep unfolding solver tailored to HSI that couples a parameter estimation module with learnable proximal updates. Extensive experiments on standard benchmarks show consistent improvements over state-of-the-art RGB-based methods in reconstruction accuracy. Code: https://github.com/psykheXX/USCTNet-Code-Implementation.git

Method: Uses a factored representation of multi-view scene geometry (depth maps, local ray maps, camera poses, metric scale factor) with transformer architecture, standardized supervision across datasets, and flexible input augmentation.

Result: Outperforms or matches specialist feed-forward models across various 3D vision tasks while offering more efficient joint training behavior.

Conclusion: MapAnything demonstrates the viability of a universal 3D reconstruction backbone that can handle multiple tasks effectively, paving the way for more unified approaches in 3D computer vision.

Abstract: We introduce MapAnything, a unified transformer-based feed-forward model that ingests one or more images along with optional geometric inputs such as camera intrinsics, poses, depth, or partial reconstructions, and then directly regresses the metric 3D scene geometry and cameras. MapAnything leverages a factored representation of multi-view scene geometry, i.e., a collection of depth maps, local ray maps, camera poses, and a metric scale factor that effectively upgrades local reconstructions into a globally consistent metric frame. Standardizing the supervision and training across diverse datasets, along with flexible input augmentation, enables MapAnything to address a broad range of 3D vision tasks in a single feed-forward pass, including uncalibrated structure-from-motion, calibrated multi-view stereo, monocular depth estimation, camera localization, depth completion, and more. We provide extensive experimental analyses and model ablations demonstrating that MapAnything outperforms or matches specialist feed-forward models while offering more efficient joint training behavior, thus paving the way toward a universal 3D reconstruction backbone.

Method: AntiPure uses two guidance mechanisms: 1) Patch-wise Frequency Guidance to reduce model influence over high-frequency components, and 2) Erroneous Timestep Guidance to disrupt denoising strategies across timesteps.

Result: AntiPure achieves minimal perceptual discrepancy and maximal distortion, outperforming other protective perturbation methods in the purification-customization workflow.

Conclusion: AntiPure effectively exposes purification vulnerabilities and provides robust protection against image misuse in diffusion models by maintaining persistent imperceptible perturbations.

Abstract: Diffusion models like Stable Diffusion have become prominent in visual synthesis tasks due to their powerful customization capabilities, which also introduce significant security risks, including deepfakes and copyright infringement. In response, a class of methods known as protective perturbation emerged, which mitigates image misuse by injecting imperceptible adversarial noise. However, purification can remove protective perturbations, thereby exposing images again to the risk of malicious forgery. In this work, we formalize the anti-purification task, highlighting challenges that hinder existing approaches, and propose a simple diagnostic protective perturbation named AntiPure. AntiPure exposes vulnerabilities of purification within the “purification-customization” workflow, owing to two guidance mechanisms: 1) Patch-wise Frequency Guidance, which reduces the model’s influence over high-frequency components in the purified image, and 2) Erroneous Timestep Guidance, which disrupts the model’s denoising strategy across different timesteps. With additional guidance, AntiPure embeds imperceptible perturbations that persist under representative purification settings, achieving effective post-customization distortion. Experiments show that, as a stress test for purification, AntiPure achieves minimal perceptual discrepancy and maximal distortion, outperforming other protective perturbation methods within the purification-customization workflow.

Method: Three core innovations: 1) Large-scale data curation pipeline with scoring/filtering strategies for captioning, OCR, QA, and video data; 2) Progressive training framework starting with SAIL-ViT vision encoder, multimodal pre-training, and thinking-fusion SFT-RL hybrid paradigm; 3) Architectural advances including sparse Mixture-of-Experts designs.

Result: Achieves state-of-the-art performance across 106 datasets, top results on challenging reasoning benchmarks (MMMU, MathVista), and ranks first among open-source models under 4B parameters on OpenCompass leaderboard.

Conclusion: SAIL-VL2 demonstrates competitive performance and serves as an efficient foundation model for the open-source multimodal community, showing strong capabilities from fine-grained perception to complex reasoning.

Abstract: We introduce SAIL-VL2, an open-suite vision-language foundation model (LVM) for comprehensive multimodal understanding and reasoning. As the successor to SAIL-VL, SAIL-VL2 achieves state-of-the-art performance at the 2B and 8B parameter scales across diverse image and video benchmarks, demonstrating strong capabilities from fine-grained perception to complex reasoning. Its effectiveness is driven by three core innovations. First, a large-scale data curation pipeline with scoring and filtering strategies enhances both quality and distribution across captioning, OCR, QA, and video data, improving training efficiency. Second, a progressive training framework begins with a powerful pre-trained vision encoder (SAIL-ViT), advances through multimodal pre-training, and culminates in a thinking-fusion SFT-RL hybrid paradigm that systematically strengthens model capabilities. Third, architectural advances extend beyond dense LLMs to efficient sparse Mixture-of-Experts (MoE) designs. With these contributions, SAIL-VL2 demonstrates competitive performance across 106 datasets and achieves state-of-the-art results on challenging reasoning benchmarks such as MMMU and MathVista. Furthermore, on the OpenCompass leaderboard, SAIL-VL2-2B ranks first among officially released open-source models under the 4B parameter scale, while serving as an efficient and extensible foundation for the open-source multimodal community.

Method: Created a synthetic dataset with 12,328 structured annotated scenes, 57,440 rooms, and 4.7M photorealistic 2D renderings. Developed SpatialGen, a multi-view multi-modal diffusion model that takes a 3D layout and reference image to synthesize appearance, geometry, and semantic maps from arbitrary viewpoints while maintaining spatial consistency.

Result: SpatialGen consistently generates superior results compared to previous methods, producing realistic and semantically consistent 3D indoor scenes with preserved spatial consistency across modalities.

Conclusion: The introduced dataset and SpatialGen model advance indoor scene understanding and generation. The authors are open-sourcing both the data and models to empower the research community.

Abstract: Creating high-fidelity 3D models of indoor environments is essential for applications in design, virtual reality, and robotics. However, manual 3D modeling remains time-consuming and labor-intensive. While recent advances in generative AI have enabled automated scene synthesis, existing methods often face challenges in balancing visual quality, diversity, semantic consistency, and user control. A major bottleneck is the lack of a large-scale, high-quality dataset tailored to this task. To address this gap, we introduce a comprehensive synthetic dataset, featuring 12,328 structured annotated scenes with 57,440 rooms, and 4.7M photorealistic 2D renderings. Leveraging this dataset, we present SpatialGen, a novel multi-view multi-modal diffusion model that generates realistic and semantically consistent 3D indoor scenes. Given a 3D layout and a reference image (derived from a text prompt), our model synthesizes appearance (color image), geometry (scene coordinate map), and semantic (semantic segmentation map) from arbitrary viewpoints, while preserving spatial consistency across modalities. SpatialGen consistently generates superior results to previous methods in our experiments. We are open-sourcing our data and models to empower the community and advance the field of indoor scene understanding and generation.

Method: Developed an improved synthetic data generation pipeline incorporating the commonly omitted forward scattering term and nonuniform medium modeling. Collected the BUCKET dataset with real turbid footage and corresponding reference images under controlled turbidity conditions.

Result: Demonstrated qualitative improvements over the reference model, especially under increasing turbidity, with 82.5% selection rate by survey participants.

Conclusion: The proposed pipeline effectively captures complex visibility loss in turbid underwater environments, providing better synthetic data generation for such challenging conditions.

Abstract: In recent years, the underwater image formation model has found extensive use in the generation of synthetic underwater data. Although many approaches focus on scenes primarily affected by discoloration, they often overlook the model’s ability to capture the complex, distance-dependent visibility loss present in highly turbid environments. In this work, we propose an improved synthetic data generation pipeline that includes the commonly omitted forward scattering term, while also considering a nonuniform medium. Additionally, we collected the BUCKET dataset under controlled turbidity conditions to acquire real turbid footage with the corresponding reference images. Our results demonstrate qualitative improvements over the reference model, particularly under increasing turbidity, with a selection rate of 82.5% by survey participants. Data and code can be accessed on the project page: vap.aau.dk/sea-ing-through-scattered-rays.

Method: Three key components: (1) Patch-wise Tracking Filters for robust sparse hinge-like relations, (2) Outlier-aware Joint Optimization with adaptive down-weighting of moving outliers, (3) Two-stage Optimization Strategy balancing Softplus limits and convex minima for stability and speed.

Result: Experiments on 4 real-world datasets (NeRF-DS, DAVIS, iPhone, TUM-dynamics) and 1 synthetic dataset (MPI-Sintel) show ROS-Cam estimates camera parameters more efficiently and accurately than COLMAP using only single RGB video supervision.

Conclusion: ROS-Cam provides a more practical and effective solution for camera parameter optimization in dynamic scenes by eliminating the need for additional supervision while improving both accuracy and efficiency compared to existing methods.

Abstract: Although COLMAP has long remained the predominant method for camera parameter optimization in static scenes, it is constrained by its lengthy runtime and reliance on ground truth (GT) motion masks for application to dynamic scenes. Many efforts attempted to improve it by incorporating more priors as supervision such as GT focal length, motion masks, 3D point clouds, camera poses, and metric depth, which, however, are typically unavailable in casually captured RGB videos. In this paper, we propose a novel method for more accurate and efficient camera parameter optimization in dynamic scenes solely supervised by a single RGB video, dubbed ROS-Cam. Our method consists of three key components: (1) Patch-wise Tracking Filters, to establish robust and maximally sparse hinge-like relations across the RGB video. (2) Outlier-aware Joint Optimization, for efficient camera parameter optimization by adaptive down-weighting of moving outliers, without reliance on motion priors. (3) A Two-stage Optimization Strategy, to enhance stability and optimization speed by a trade-off between the Softplus limits and convex minima in losses. We visually and numerically evaluate our camera estimates. To further validate accuracy, we feed the camera estimates into a 4D reconstruction method and assess the resulting 3D scenes, and rendered 2D RGB and depth maps. We perform experiments on 4 real-world datasets (NeRF-DS, DAVIS, iPhone, and TUM-dynamics) and 1 synthetic dataset (MPI-Sintel), demonstrating that our method estimates camera parameters more efficiently and accurately with a single RGB video as the only supervision.

Method: Built a large-scale dataset via a closed-loop pipeline uniting automated agents with human experts, spanning 6 operating systems and 3 task domains. Trained ScaleCUA model on this scaled-up data.

Result: Achieved strong gains over baselines (+26.6 on WebArena-Lite-v2, +10.7 on ScreenSpot-Pro) and set new SOTA results (94.4% on MMBench-GUI L1-Hard, 60.6% on OSWorld-G, 47.4% on WebArena-Lite-v2).

Conclusion: Data-driven scaling is powerful for developing general-purpose computer use agents. The authors will release data, models, and code to advance future research.

Abstract: Vision-Language Models (VLMs) have enabled computer use agents (CUAs) that operate GUIs autonomously, showing great potential, yet progress is limited by the lack of large-scale, open-source computer use data and foundation models. In this work, we introduce ScaleCUA, a step toward scaling open-source CUAs. It offers a large-scale dataset spanning 6 operating systems and 3 task domains, built via a closed-loop pipeline uniting automated agents with human experts. Trained on this scaled-up data, ScaleCUA can operate seamlessly across platforms. Specifically, it delivers strong gains over baselines (+26.6 on WebArena-Lite-v2, +10.7 on ScreenSpot-Pro) and sets new state-of-the-art results (94.4% on MMBench-GUI L1-Hard, 60.6% on OSWorld-G, 47.4% on WebArena-Lite-v2). These findings underscore the power of data-driven scaling for general-purpose computer use agents. We will release data, models, and code to advance future research: https://github.com/OpenGVLab/ScaleCUA.

Method: MICA coordinates five role-specialized language agents with safety auditing, uses Adaptive Step Fusion (ASF) for robust step understanding by blending expert reasoning with speech feedback adaptation, and establishes a multi-agent coordination benchmark for industrial assistance evaluation.

Result: Experiments show MICA consistently improves task success, reliability, and responsiveness over baseline structures while remaining deployable on practical offline hardware.

Conclusion: MICA represents a step toward deployable, privacy-preserving multi-agent assistants for dynamic factory environments, with source code made publicly available.

Abstract: Industrial workflows demand adaptive and trustworthy assistance that can operate under limited computing, connectivity, and strict privacy constraints. In this work, we present MICA (Multi-Agent Industrial Coordination Assistant), a perception-grounded and speech-interactive system that delivers real-time guidance for assembly, troubleshooting, part queries, and maintenance. MICA coordinates five role-specialized language agents, audited by a safety checker, to ensure accurate and compliant support. To achieve robust step understanding, we introduce Adaptive Step Fusion (ASF), which dynamically blends expert reasoning with online adaptation from natural speech feedback. Furthermore, we establish a new multi-agent coordination benchmark across representative task categories and propose evaluation metrics tailored to industrial assistance, enabling systematic comparison of different coordination topologies. Our experiments demonstrate that MICA consistently improves task success, reliability, and responsiveness over baseline structures, while remaining deployable on practical offline hardware. Together, these contributions highlight MICA as a step toward deployable, privacy-preserving multi-agent assistants for dynamic factory environments. The source code will be made publicly available at https://github.com/Kratos-Wen/MICA.

Method: A two-phase pipeline that first constructs a dynamic programming table and then reconstructs the solution from it, using dynamic programming supervision to model intermediate states rather than directly predicting the optimal subset from inputs.

Result: The neural algorithmic reasoner achieves better generalization to larger Knapsack problem instances compared to a direct-prediction baseline that attempts to select the optimal subset only from problem inputs.

Conclusion: Modeling intermediate states through dynamic programming supervision in a two-phase approach is more effective for neural algorithmic reasoning on Knapsack problems, providing superior generalization capabilities over direct-prediction methods.

Abstract: Neural algorithmic reasoning (NAR) is a growing field that aims to embed algorithmic logic into neural networks by imitating classical algorithms. In this extended abstract, we detail our attempt to build a neural algorithmic reasoner that can solve Knapsack, a pseudo-polynomial problem bridging classical algorithms and combinatorial optimisation, but omitted in standard NAR benchmarks. Our neural algorithmic reasoner is designed to closely follow the two-phase pipeline for the Knapsack problem, which involves first constructing the dynamic programming table and then reconstructing the solution from it. The approach, which models intermediate states through dynamic programming supervision, achieves better generalization to larger problem instances than a direct-prediction baseline that attempts to select the optimal subset only from the problem inputs.

Method: Evaluation and analysis of MetaLight, a Meta Reinforcement Learning approach, to assess its performance and robustness in handling distribution shifts in traffic data.

Result: MetaLight produces reasonably good results under certain conditions but performs poorly under others, with errors reaching up to 22%, highlighting insufficient robustness.

Conclusion: Meta RL schemes like MetaLight are often not robust enough and can pose major reliability problems, suggesting the need for more reliable solutions in AI-based traffic control systems.

Abstract: The use of Machine Learning (ML) and Artificial Intelligence (AI) in smart transportation networks has increased significantly in the last few years. Among these ML and AI approaches, Reinforcement Learning (RL) has been shown to be a very promising approach by several authors. However, a problem with using Reinforcement Learning in Traffic Signal Control is the reliability of the trained RL agents due to the dynamically changing distribution of the input data with respect to the distribution of the data used for training. This presents a major challenge and a reliability problem for the trained network of AI agents and could have very undesirable and even detrimental consequences if a suitable solution is not found. Several researchers have tried to address this problem using different approaches. In particular, Meta Reinforcement Learning (Meta RL) promises to be an effective solution. In this paper, we evaluate and analyze a state-of-the-art Meta RL approach called MetaLight and show that, while under certain conditions MetaLight can indeed lead to reasonably good results, under some other conditions it might not perform well (with errors of up to 22%), suggesting that Meta RL schemes are often not robust enough and can even pose major reliability problems.

Method: Uses transformer-based embeddings and cosine similarity to generate relevant keywords from input paper title/abstract, fetches papers from open access repository, and ranks them by semantic closeness. Evaluated three embedding models with statistical thresholding for filtering.

Result: The proposed system shows promise as a scalable tool for preliminary research and exploratory analysis, despite lacking heuristic feedback or ground truth relevance labels.

Conclusion: The automated pipeline demonstrates effectiveness for literature reviews using semantic similarity, offering a practical alternative to more complex traditional methods.

Abstract: We propose an automated pipeline for performing literature reviews using semantic similarity. Unlike traditional systematic review systems or optimization based methods, this work emphasizes minimal overhead and high relevance by using transformer based embeddings and cosine similarity. By providing a paper title and abstract, it generates relevant keywords, fetches relevant papers from open access repository, and ranks them based on their semantic closeness to the input. Three embedding models were evaluated. A statistical thresholding approach is then applied to filter relevant papers, enabling an effective literature review pipeline. Despite the absence of heuristic feedback or ground truth relevance labels, the proposed system shows promise as a scalable and practical tool for preliminary research and exploratory analysis.

Method: Conducted controlled experiments using automated process modeling tasks, creating deliberate conflicts between provided evidence and LLM background knowledge, testing both standard and atypical process structures.

Result: LLMs demonstrated tendency to prioritize their internal knowledge over explicit source evidence, leading to contradictions and reduced fidelity to provided inputs.

Conclusion: The study provides a methodology for assessing LLM reliability issues and emphasizes the need for rigorous validation of AI-generated artifacts in evidence-based domains.

Abstract: The utility of Large Language Models (LLMs) in analytical tasks is rooted in their vast pre-trained knowledge, which allows them to interpret ambiguous inputs and infer missing information. However, this same capability introduces a critical risk of what we term knowledge-driven hallucination: a phenomenon where the model’s output contradicts explicit source evidence because it is overridden by the model’s generalized internal knowledge. This paper investigates this phenomenon by evaluating LLMs on the task of automated process modeling, where the goal is to generate a formal business process model from a given source artifact. The domain of Business Process Management (BPM) provides an ideal context for this study, as many core business processes follow standardized patterns, making it likely that LLMs possess strong pre-trained schemas for them. We conduct a controlled experiment designed to create scenarios with deliberate conflict between provided evidence and the LLM’s background knowledge. We use inputs describing both standard and deliberately atypical process structures to measure the LLM’s fidelity to the provided evidence. Our work provides a methodology for assessing this critical reliability issue and raises awareness of the need for rigorous validation of AI-generated artifacts in any evidence-based domain.

Method: Framework integrates curated golden datasets of expert annotations, silver datasets from controlled behavioral mutation, and an LLM-based Agent Judge that scores agents and provides prescriptions embedded in a vectorized recommendation map.

Result: Applied to a multi-agent recruiter-assistant system, the framework uncovered latent cognitive failures (biased phrasing, extraction drift, tool misrouting) while steering agents toward expert-level reasoning and style.

Conclusion: Establishes foundation for standardized, reproducible expert behavior transfer in stochastic, tool-augmented LLM agents, enabling active expert system refinement beyond static evaluation.

Abstract: The rapid evolution of neural architectures - from multilayer perceptrons to large-scale Transformer-based models - has enabled language models (LLMs) to exhibit emergent agentic behaviours when equipped with memory, planning, and external tool use. However, their inherent stochasticity and multi-step decision processes render classical evaluation methods inadequate for diagnosing agentic performance. This work introduces a diagnostic framework for expert systems that not only evaluates but also facilitates the transfer of expert behaviour into LLM-powered agents. The framework integrates (i) curated golden datasets of expert annotations, (ii) silver datasets generated through controlled behavioural mutation, and (iii) an LLM-based Agent Judge that scores and prescribes targeted improvements. These prescriptions are embedded into a vectorized recommendation map, allowing expert interventions to propagate as reusable improvement trajectories across multiple system instances. We demonstrate the framework on a multi-agent recruiter-assistant system, showing that it uncovers latent cognitive failures - such as biased phrasing, extraction drift, and tool misrouting - while simultaneously steering agents toward expert-level reasoning and style. The results establish a foundation for standardized, reproducible expert behaviour transfer in stochastic, tool-augmented LLM agents, moving beyond static evaluation to active expert system refinement.

Method: FragmentRetro leverages fragmentation algorithms (BRICS and r-BRICS) combined with stock-aware exploration and pattern fingerprint screening to recursively combine molecular fragments and verify their presence in a building block set.

Result: FragmentRetro achieves O(h²) complexity compared to tree search’s O(bʰ) and DirectMultiStep’s O(h⁶). Evaluations on PaRoutes, USPTO-190, and natural products show high solved rates with competitive runtime, including cases where tree search fails.

Conclusion: FragmentRetro establishes itself as a powerful foundational component for scalable and automated synthesis planning, efficiently identifying fragment-based solutions with significant computational advantages.

Abstract: Retrosynthesis, the process of deconstructing a target molecule into simpler precursors, is crucial for computer-aided synthesis planning (CASP). Widely adopted tree-search methods often suffer from exponential computational complexity. In this work, we introduce FragmentRetro, a novel retrosynthetic method that leverages fragmentation algorithms, specifically BRICS and r-BRICS, combined with stock-aware exploration and pattern fingerprint screening to achieve quadratic complexity. FragmentRetro recursively combines molecular fragments and verifies their presence in a building block set, providing sets of fragment combinations as retrosynthetic solutions. We present the first formal computational analysis of retrosynthetic methods, showing that tree search exhibits exponential complexity $O(b^h)$, DirectMultiStep scales as $O(h^6)$, and FragmentRetro achieves $O(h^2)$, where $h$ represents the number of heavy atoms in the target molecule and $b$ is the branching factor for tree search. Evaluations on PaRoutes, USPTO-190, and natural products demonstrate that FragmentRetro achieves high solved rates with competitive runtime, including cases where tree search fails. The method benefits from fingerprint screening, which significantly reduces substructure matching complexity. While FragmentRetro focuses on efficiently identifying fragment-based solutions rather than full reaction pathways, its computational advantages and ability to generate strategic starting candidates establish it as a powerful foundational component for scalable and automated synthesis planning.

Method: Developed a multi-agent framework with three specialist sub-agents: data science agent for time-series analysis, health domain expert for personalized insights, and health coach agent for guidance using psychological strategies. Conducted extensive user research and evaluations across 10 benchmark tasks.

Result: The system was evaluated through automated and human assessments involving 7,000+ annotations and 1,100+ hours of expert effort, representing the most comprehensive health agent evaluation to date.

Conclusion: The work establishes a strong foundation for accessible personal health agents that can dynamically address individual health needs through personalized interactions.

Abstract: Health is a fundamental pillar of human wellness, and the rapid advancements in large language models (LLMs) have driven the development of a new generation of health agents. However, the application of health agents to fulfill the diverse needs of individuals in daily non-clinical settings is underexplored. In this work, we aim to build a comprehensive personal health agent that is able to reason about multimodal data from everyday consumer wellness devices and common personal health records, and provide personalized health recommendations. To understand end-users’ needs when interacting with such an assistant, we conducted an in-depth analysis of web search and health forum queries, alongside qualitative insights from users and health experts gathered through a user-centered design process. Based on these findings, we identified three major categories of consumer health needs, each of which is supported by a specialist sub-agent: (1) a data science agent that analyzes personal time-series wearable and health record data, (2) a health domain expert agent that integrates users’ health and contextual data to generate accurate, personalized insights, and (3) a health coach agent that synthesizes data insights, guiding users using a specified psychological strategy and tracking users’ progress. Furthermore, we propose and develop the Personal Health Agent (PHA), a multi-agent framework that enables dynamic, personalized interactions to address individual health needs. To evaluate each sub-agent and the multi-agent system, we conducted automated and human evaluations across 10 benchmark tasks, involving more than 7,000 annotations and 1,100 hours of effort from health experts and end-users. Our work represents the most comprehensive evaluation of a health agent to date and establishes a strong foundation towards the futuristic vision of a personal health agent accessible to everyone.

Method: Uses covert actions as a proxy for scheming, tests anti-scheming interventions on out-of-distribution tasks, evaluates situational awareness through chain-of-thought analysis, and stress-tests deliberative alignment across 180+ environments.

Result: Deliberative alignment reduces covert action rates (from 13% to 0.4%) but doesn’t fully eliminate them. Models show situational awareness during evaluations, which decreases covert behavior. The mitigation largely stops agents from pursuing pre-trained hidden goals but misbehavior persists after red-teaming.

Conclusion: While deliberative alignment shows promise, reductions in scheming may be partially driven by situational awareness. Future research should address deceptive alignment and develop mitigations that work when models no longer reason in standard English.

Abstract: Highly capable AI systems could secretly pursue misaligned goals – what we call “scheming”. Because a scheming AI would deliberately try to hide its misaligned goals and actions, measuring and mitigating scheming requires different strategies than are typically used in ML. We propose that assessing anti-scheming interventions requires at least (1) testing propensity to scheme on far out-of-distribution (OOD) tasks, (2) evaluating whether lack of scheming is driven by situational awareness, and (3) checking for robustness to pre-existing misaligned goals. We use a broad category of “covert actions” – such as secretly breaking rules or intentionally underperforming in tests – as a proxy for scheming, and design evaluations for covert actions. We then stress-test deliberative alignment as a case study for anti-scheming. Across 26 OOD evaluations (180+ environments), deliberative alignment reduces covert action rates (OpenAI o3: 13%->0.4%) but does not fully eliminate them. Our mitigation is also able to largely stop agents from pursuing a hidden goal previously trained into the model, but we still find misbehavior after additional red-teaming. We find that models’ chain-of-thought (CoT) often demonstrates awareness of being evaluated for alignment, and show causal evidence that this awareness decreases covert behavior, while unawareness increases it. Therefore, we cannot exclude that the observed reductions in covert action rates are at least partially driven by situational awareness. While we rely on human-legible CoT for training, studying situational awareness, and demonstrating clear evidence of misalignment, our ability to rely on this degrades as models continue to depart from reasoning in standard English. We encourage research into alignment mitigations for scheming and their assessment, especially for the adversarial case of deceptive alignment, which this paper does not address.

Method: Three-step approach: 1) Pre-trained Drain log parsing with multi-level data filtering for log compression; 2) Dual anomaly detection combining Isolation Forest unsupervised learning with status code validation; 3) Statistical symmetry ratio filtering with two-stage LLM analysis for full-stack phenomenon summarization.

Result: The system achieves a final score of 50.71 in complex microservice fault scenarios, with comprehensive ablation studies validating the complementary value of each modal data and system architecture effectiveness.

Conclusion: MicroRCA-Agent demonstrates superior performance in microservice root cause analysis through its innovative multimodal fusion approach and structured LLM-based reasoning capabilities.

Abstract: This paper presents MicroRCA-Agent, an innovative solution for microservice root cause analysis based on large language model agents, which constructs an intelligent fault root cause localization system with multimodal data fusion. The technical innovations are embodied in three key aspects: First, we combine the pre-trained Drain log parsing algorithm with multi-level data filtering mechanism to efficiently compress massive logs into high-quality fault features. Second, we employ a dual anomaly detection approach that integrates Isolation Forest unsupervised learning algorithms with status code validation to achieve comprehensive trace anomaly identification. Third, we design a statistical symmetry ratio filtering mechanism coupled with a two-stage LLM analysis strategy to enable full-stack phenomenon summarization across node-service-pod hierarchies. The multimodal root cause analysis module leverages carefully designed cross-modal prompts to deeply integrate multimodal anomaly information, fully exploiting the cross-modal understanding and logical reasoning capabilities of large language models to generate structured analysis results encompassing fault components, root cause descriptions, and reasoning trace. Comprehensive ablation studies validate the complementary value of each modal data and the effectiveness of the system architecture. The proposed solution demonstrates superior performance in complex microservice fault scenarios, achieving a final score of 50.71. The code has been released at: https://github.com/tangpan360/MicroRCA-Agent.

Method: Three core contributions: (1) CCrepair dataset construction via generate-and-verify pipeline, (2) RL paradigm with hybrid reward signal focusing on semantic quality, (3) two-stage evaluation system using LLM-as-a-Judge validated against human experts.

Result: RL-trained Qwen2.5-1.5B-Instruct model achieved performance comparable to Qwen2.5-14B-Instruct model, demonstrating training efficiency and effectiveness.

Conclusion: Provides valuable dataset and effective paradigm for training robust compilation repair models, advancing practical automated programming assistants.

Abstract: The automated repair of C++ compilation errors presents a significant challenge, the resolution of which is critical for developer productivity. Progress in this domain is constrained by two primary factors: the scarcity of large-scale, high-fidelity datasets and the limitations of conventional supervised methods, which often fail to generate semantically correct patches.This paper addresses these gaps by introducing a comprehensive framework with three core contributions. First, we present CCrepair, a novel, large-scale C++ compilation error dataset constructed through a sophisticated generate-and-verify pipeline. Second, we propose a Reinforcement Learning (RL) paradigm guided by a hybrid reward signal, shifting the focus from mere compilability to the semantic quality of the fix. Finally, we establish the robust, two-stage evaluation system providing this signal, centered on an LLM-as-a-Judge whose reliability has been rigorously validated against the collective judgments of a panel of human experts. This integrated approach aligns the training objective with generating high-quality, non-trivial patches that are both syntactically and semantically correct. The effectiveness of our approach was demonstrated experimentally. Our RL-trained Qwen2.5-1.5B-Instruct model achieved performance comparable to a Qwen2.5-14B-Instruct model, validating the efficiency of our training paradigm. Our work provides the research community with a valuable new dataset and a more effective paradigm for training and evaluating robust compilation repair models, paving the way for more practical and reliable automated programming assistants.

Method: The authors conducted a literature review to explore RPA-ML connections and organized the joint concept of intelligent RPA into a taxonomy comprising two meta-characteristics: RPA-ML integration and RPA-ML interaction.

Result: The taxonomy includes eight dimensions: architecture and ecosystem, capabilities, data basis, intelligence level, technical depth of integration, deployment environment, lifecycle phase, and user-robot relation.

Conclusion: The proposed taxonomy provides a structured framework for understanding and implementing intelligent RPA systems that combine traditional RPA with machine learning capabilities to automate more complex business processes.

Abstract: Robotic process automation (RPA) is a lightweight approach to automating business processes using software robots that emulate user actions at the graphical user interface level. While RPA has gained popularity for its cost-effective and timely automation of rule-based, well-structured tasks, its symbolic nature has inherent limitations when approaching more complex tasks currently performed by human agents. Machine learning concepts enabling intelligent RPA provide an opportunity to broaden the range of automatable tasks. In this paper, we conduct a literature review to explore the connections between RPA and machine learning and organize the joint concept intelligent RPA into a taxonomy. Our taxonomy comprises the two meta-characteristics RPA-ML integration and RPA-ML interaction. Together, they comprise eight dimensions: architecture and ecosystem, capabilities, data basis, intelligence level, and technical depth of integration as well as deployment environment, lifecycle phase, and user-robot relation.

Method: Developed a Knowledge Representation (KR) model and Knowledge Management (KM) tools that enable physiology experts to capture anatomical interactions and convert high-level abstractions into detailed physiological process models.

Result: Created infrastructure that integrates with external ontologies and knowledge graphs for comprehensive physiological mapping.

Conclusion: ApiNATOMY provides an effective framework for representing and managing multiscale physiological circuit maps, enhancing research capabilities in physiological system mapping.

Abstract: We are developing infrastructure to support researchers in mapping data related to the peripheral nervous system and other physiological systems, with an emphasis on their relevance to the organs under investigation. The nervous system, a complex network of nerves and ganglia, plays a critical role in coordinating and transmitting signals throughout the body. To aid in this, we have created ApiNATOMY, a framework for the topological and semantic representation of multiscale physiological circuit maps. ApiNATOMY integrates a Knowledge Representation (KR) model and a suite of Knowledge Management (KM) tools. The KR model enables physiology experts to easily capture interactions between anatomical entities, while the KM tools help modelers convert high-level abstractions into detailed models of physiological processes, which can be integrated with external ontologies and knowledge graphs.

Method: CLIMB uses global semantic clustering to distill core occupations, then employs a reflection-based multi-agent system to iteratively build a coherent hierarchy.

Result: On three diverse real-world datasets, CLIMB produces taxonomies that are more coherent and scalable than existing methods and successfully captures unique regional characteristics.

Conclusion: CLIMB provides a fully automated solution for creating high-quality, data-driven occupation taxonomies that address limitations of both manual curation and existing automated approaches.

Abstract: Creating robust occupation taxonomies, vital for applications ranging from job recommendation to labor market intelligence, is challenging. Manual curation is slow, while existing automated methods are either not adaptive to dynamic regional markets (top-down) or struggle to build coherent hierarchies from noisy data (bottom-up). We introduce CLIMB (CLusterIng-based Multi-agent taxonomy Builder), a framework that fully automates the creation of high-quality, data-driven taxonomies from raw job postings. CLIMB uses global semantic clustering to distill core occupations, then employs a reflection-based multi-agent system to iteratively build a coherent hierarchy. On three diverse, real-world datasets, we show that CLIMB produces taxonomies that are more coherent and scalable than existing methods and successfully capture unique regional characteristics. We release our code and datasets at https://anonymous.4open.science/r/CLIMB.

Method: Comparative analysis of rule-based vs. data-driven systems, examining their properties, and proposing a basic evaluation framework for key characteristics.

Result: Rule-based systems offer interpretability and deterministic behavior but lack scalability; data-driven systems excel in anomaly detection but face explainability challenges; hybrid solutions show promise.

Conclusion: Future industrial monitoring should combine rule-based transparency with data-driven analytical power through intelligent hybrid systems for enhanced resilience and efficiency.

Abstract: Industrial monitoring systems, especially when deployed in Industry 4.0 environments, are experiencing a shift in paradigm from traditional rule-based architectures to data-driven approaches leveraging machine learning and artificial intelligence. This study presents a comparison between these two methodologies, analyzing their respective strengths, limitations, and application scenarios, and proposes a basic framework to evaluate their key properties. Rule-based systems offer high interpretability, deterministic behavior, and ease of implementation in stable environments, making them ideal for regulated industries and safety-critical applications. However, they face challenges with scalability, adaptability, and performance in complex or evolving contexts. Conversely, data-driven systems excel in detecting hidden anomalies, enabling predictive maintenance and dynamic adaptation to new conditions. Despite their high accuracy, these models face challenges related to data availability, explainability, and integration complexity. The paper suggests hybrid solutions as a possible promising direction, combining the transparency of rule-based logic with the analytical power of machine learning. Our hypothesis is that the future of industrial monitoring lies in intelligent, synergic systems that leverage both expert knowledge and data-driven insights. This dual approach enhances resilience, operational efficiency, and trust, paving the way for smarter and more flexible industrial environments.

Method: Developed EHR-MCP framework with custom MCP tools integrated with hospital EHR database, using GPT-4.1 through LangGraph ReAct agent. Tested six infection control team tasks on eight patients, comparing against physician-generated gold standards.

Result: LLM consistently selected correct MCP tools, achieving near-perfect accuracy except for complex time-dependent tasks. Most errors from incorrect arguments or tool result misinterpretation. Responses were reliable but risked exceeding context window with long/repetitive data.

Conclusion: LLMs can successfully retrieve clinical data from EHRs via MCP in real hospital settings, providing secure infrastructure for hospital AI agents. Future work should extend to reasoning, generation, and clinical impact assessment for effective AI integration into clinical practice.

Abstract: Background: Large language models (LLMs) show promise in medicine, but their deployment in hospitals is limited by restricted access to electronic health record (EHR) systems. The Model Context Protocol (MCP) enables integration between LLMs and external tools. Objective: To evaluate whether an LLM connected to an EHR database via MCP can autonomously retrieve clinically relevant information in a real hospital setting. Methods: We developed EHR-MCP, a framework of custom MCP tools integrated with the hospital EHR database, and used GPT-4.1 through a LangGraph ReAct agent to interact with it. Six tasks were tested, derived from use cases of the infection control team (ICT). Eight patients discussed at ICT conferences were retrospectively analyzed. Agreement with physician-generated gold standards was measured. Results: The LLM consistently selected and executed the correct MCP tools. Except for two tasks, all tasks achieved near-perfect accuracy. Performance was lower in the complex task requiring time-dependent calculations. Most errors arose from incorrect arguments or misinterpretation of tool results. Responses from EHR-MCP were reliable, though long and repetitive data risked exceeding the context window. Conclusions: LLMs can retrieve clinical data from an EHR via MCP tools in a real hospital setting, achieving near-perfect performance in simple tasks while highlighting challenges in complex ones. EHR-MCP provides an infrastructure for secure, consistent data access and may serve as a foundation for hospital AI agents. Future work should extend beyond retrieval to reasoning, generation, and clinical impact assessment, paving the way for effective integration of generative AI into clinical practice.

Method: Uses a fine-tuned language model to automatically convert prompts into tuple-based structured information, which is then seamlessly integrated into T2I pipelines.

Result: Substantial improvements in spatial accuracy without compromising overall image quality (Inception Score), with automatically generated tuples matching human-crafted quality.

Conclusion: The structured information approach provides a practical and portable solution to enhance spatial relationships in T2I generation, addressing a key limitation of current large-scale generative systems.

Abstract: Text-to-image (T2I) generation has advanced rapidly, yet faithfully capturing spatial relationships described in natural language prompts remains a major challenge. Prior efforts have addressed this issue through prompt optimization, spatially grounded generation, and semantic refinement. This work introduces a lightweight approach that augments prompts with tuple-based structured information, using a fine-tuned language model for automatic conversion and seamless integration into T2I pipelines. Experimental results demonstrate substantial improvements in spatial accuracy, without compromising overall image quality as measured by Inception Score. Furthermore, the automatically generated tuples exhibit quality comparable to human-crafted tuples. This structured information provides a practical and portable solution to enhance spatial relationships in T2I generation, addressing a key limitation of current large-scale generative systems.

Method: Introduces ASAC module that uses Vector-Quantized Variational AutoEncoder (VQVAE) as both attention abstractor and controller, integrated into transformer architectures to explicitly model and manage attention allocation.

Result: Demonstrated effectiveness in vision and NLP domains: improved classification accuracy, accelerated learning, enhanced robustness to noise and out-of-distribution data, better multi-task performance, and increased resilience to adversarial attacks.

Conclusion: ASAC successfully connects cognitive science principles with machine learning, showing that explicit attention modeling can optimize attention mechanisms in AI systems for improved efficiency and performance across various tasks.

Abstract: Attention mechanisms have become integral in AI, significantly enhancing model performance and scalability by drawing inspiration from human cognition. Concurrently, the Attention Schema Theory (AST) in cognitive science posits that individuals manage their attention by creating a model of the attention itself, effectively allocating cognitive resources. Inspired by AST, we introduce ASAC (Attention Schema-based Attention Control), which integrates the attention schema concept into artificial neural networks. Our initial experiments focused on embedding the ASAC module within transformer architectures. This module employs a Vector-Quantized Variational AutoEncoder (VQVAE) as both an attention abstractor and controller, facilitating precise attention management. By explicitly modeling attention allocation, our approach aims to enhance system efficiency. We demonstrate ASAC’s effectiveness in both the vision and NLP domains, highlighting its ability to improve classification accuracy and expedite the learning process. Our experiments with vision transformers across various datasets illustrate that the attention controller not only boosts classification accuracy but also accelerates learning. Furthermore, we have demonstrated the model’s robustness and generalization capabilities across noisy and out-of-distribution datasets. In addition, we have showcased improved performance in multi-task settings. Quick experiments reveal that the attention schema-based module enhances resilience to adversarial attacks, optimizes attention to improve learning efficiency, and facilitates effective transfer learning and learning from fewer examples. These promising results establish a connection between cognitive science and machine learning, shedding light on the efficient utilization of attention mechanisms in AI systems.

Method: Cognitive-logs store episodic memory as graphical networks using actions (verbs) and participants connected by morphisms. The design is based on cognitive linguistics principles, with logical reasoning performed by comparing causal chains using category theory operations.

Result: The proposed cognitive-logs format enables various inferences including planning, comprehension, and hierarchical story abstractions through category theory-based operations on episodic memories.

Conclusion: Cognitive-logs provide a scalable database approach (up to petabyte scale) for AI systems that combines human-like thinking with machine precision, serving as both an AI framework and a model of human cognition.

Abstract: This study presents data format of episodic memory for artificial intelligence and cognitive science. The data format, named cognitive-logs, enables rigour and flexible logical reasoning. Cognitive-logs consist of a set of relational and graph databases. Cognitive-logs store an episodic memory as a graphical network that consist of “actions” represented by verbs in natural languages and “participants” who perform the actions. These objects are connected by arrows (morphisms) that bind each action to its participant and bind causes and effects. The design principle of cognitive-logs refers cognitive sciences especially in cognitive linguistics. Logical reasoning is the processes of comparing causal chains in episodic memories with known rules which are also recorded in the cognitive-logs. Operations based on category theory enable such comparisons between episodic memories or scenarios. These operations represent various inferences including planning, comprehensions, and hierarchical abstractions of stories. The goal of this study is to develop a database-driven artificial intelligence that thinks like a human but possesses the accuracy and rigour of a machine. The vast capacities of databases (up to petabyte scales in current technologies) enable the artificial intelligence to store a greater volume of knowledge than neural-network based artificial intelligences. Cognitive-logs also serve as a model of human cognition mind activities.

Method: A DRL architecture with a semantic clustering module that combines feature dimensionality reduction with online clustering, integrated directly into the training pipeline.

Result: Experimental validation shows the module effectively reveals semantic clustering properties in DRL and enables new analytical methods for understanding policy hierarchies and feature space organization.

Conclusion: The proposed semantic clustering module successfully improves DRL interpretability and provides deeper insights into semantic organization without the drawbacks of previous methods.

Abstract: In this paper, we explore semantic clustering properties of deep reinforcement learning (DRL) to improve its interpretability and deepen our understanding of its internal semantic organization. In this context, semantic clustering refers to the ability of neural networks to cluster inputs based on their semantic similarity in the internal space. We propose a DRL architecture that incorporates a novel semantic clustering module that combines feature dimensionality reduction with online clustering. This module integrates seamlessly into the DRL training pipeline, addressing the instability of t-SNE and eliminating the need for extensive manual annotation inherent to prior semantic analysis methods. We experimentally validate the effectiveness of the proposed module and demonstrate its ability to reveal semantic clustering properties within DRL. Furthermore, we introduce new analytical methods based on these properties to provide insights into the hierarchical structure of policies and semantic organization within the feature space.

Method: Infer user intent through trajectory-level feedback and combine it with the trained task policy using a theoretically grounded dynamic policy fusion approach that works on existing trajectories without new environment interactions.

Result: Empirical demonstrations across multiple environments show the approach consistently achieves intended tasks while adhering to user-specific needs.

Conclusion: The proposed dynamic policy fusion provides a practical zero-shot solution for aligning pre-trained RL policies with human preferences through feedback on existing trajectories.

Abstract: Deep reinforcement learning (RL) policies, although optimal in terms of task rewards, may not align with the personal preferences of human users. To ensure this alignment, a naive solution would be to retrain the agent using a reward function that encodes the user’s specific preferences. However, such a reward function is typically not readily available, and as such, retraining the agent from scratch can be prohibitively expensive. We propose a more practical approach - to adapt the already trained policy to user-specific needs with the help of human feedback. To this end, we infer the user’s intent through trajectory-level feedback and combine it with the trained task policy via a theoretically grounded dynamic policy fusion approach. As our approach collects human feedback on the very same trajectories used to learn the task policy, it does not require any additional interactions with the environment, making it a zero-shot approach. We empirically demonstrate in a number of environments that our proposed dynamic policy fusion approach consistently achieves the intended task while simultaneously adhering to user-specific needs.

Method: Uses LLM to plan solutions, soft-formalizes queries into facts and predicates using logic programming code, and simulates execution via exhaustive multi-hop search over the defined space. Computes faithfulness w.r.t. generated code and analyzes multi-hop search steps.

Result: Achieves state-of-the-art results on 7 out of 9 diverse reasoning benchmarks. Shows positive correlation between model faithfulness and overall performance. Enables pinpointing decisive factors for correct answers.

Conclusion: FLARE provides an interpretable, faithful reasoning approach that outperforms existing methods while maintaining transparency in the reasoning process without external dependencies.

Abstract: Modern Question Answering (QA) and Reasoning approaches based on Large Language Models (LLMs) commonly use prompting techniques, such as Chain-of-Thought (CoT), assuming the resulting generation will have a more granular exploration and reasoning over the question space and scope. However, such methods struggle with generating outputs that are faithful to the intermediate chain of reasoning produced by the model. On the other end of the spectrum, neuro-symbolic methods such as Faithful CoT (F-CoT) propose to combine LLMs with external symbolic solvers. While such approaches boast a high degree of faithfulness, they usually require a model trained for code generation and struggle with tasks that are ambiguous or hard to formalise strictly. We introduce $\textbf{F}$aithful $\textbf{L}$ogic-$\textbf{A}$ided $\textbf{R}$easoning and $\textbf{E}$xploration ($\textbf{FLARE}$), a novel interpretable approach for traversing the problem space using task decompositions. We use the LLM to plan a solution, soft-formalise the query into facts and predicates using a logic programming code and simulate that code execution using an exhaustive multi-hop search over the defined space. Our method allows us to compute the faithfulness of the reasoning process w.r.t. the generated code and analyse the steps of the multi-hop search without relying on external solvers. Our methods achieve SOTA results on $\mathbf{7}$ out of $\mathbf{9}$ diverse reasoning benchmarks. We also show that model faithfulness positively correlates with overall performance and further demonstrate that $\textbf{FLARE}$ allows pinpointing the decisive factors sufficient for and leading to the correct answer with optimal reasoning during the multi-hop search.

Method: Watson uses prompt attribution techniques to retroactively infer reasoning traces from fast-thinking LLM agents, working in both static and dynamic settings without altering agent behavior.

Result: Evaluation on MMLU benchmark and AutoCodeRover/OpenHands agents on SWE-bench-lite shows Watson surfaces actionable reasoning insights and supports targeted interventions for debugging and correction.

Conclusion: Watson demonstrates practical utility for improving transparency and reliability in Agentware systems through cognitive observability of LLM agent reasoning processes.

Abstract: Large language models (LLMs) are increasingly integrated into autonomous systems, giving rise to a new class of software known as Agentware, where LLM-powered agents perform complex, open-ended tasks in domains such as software engineering, customer service, and data analysis. However, their high autonomy and opaque reasoning processes pose significant challenges for traditional software observability methods. To address this, we introduce the concept of cognitive observability - the ability to recover and inspect the implicit reasoning behind agent decisions. We present Watson, a general-purpose framework for observing the reasoning processes of fast-thinking LLM agents without altering their behavior. Watson retroactively infers reasoning traces using prompt attribution techniques. We evaluate Watson in both manual debugging and automated correction scenarios across the MMLU benchmark and the AutoCodeRover and OpenHands agents on the SWE-bench-lite dataset. In both static and dynamic settings, Watson surfaces actionable reasoning insights and supports targeted interventions, demonstrating its practical utility for improving transparency and reliability in Agentware systems.

Method: Developed a mathematical approach using Möbius inversion with closed-form expressions for the redundancy lattice, applied to logic gates, cellular automata, chemical reaction networks, and transformer language models.

Result: The decomposition reveals context- and parameter-dependent distribution of causal power, showing how causal influences are shared and combined among multiple variables.

Conclusion: The framework provides new insights into complex systems with applications in legal/AI responsibility attribution, biological networks, and climate models.

Abstract: We introduce a novel framework for decomposing interventional causal effects into synergistic, redundant, and unique components, building on the intuition of Partial Information Decomposition (PID) and the principle of M"obius inversion. While recent work has explored a similar decomposition of an observational measure, we argue that a proper causal decomposition must be interventional in nature. We develop a mathematical approach that systematically quantifies how causal power is distributed among variables in a system, using a recently derived closed-form expression for the M"obius function of the redundancy lattice. The formalism is then illustrated by decomposing the causal power in logic gates, cellular automata, chemical reaction networks, and a transformer language model. Our results reveal how the distribution of causal power can be context- and parameter-dependent. The decomposition provides new insights into complex systems by revealing how causal influences are shared and combined among multiple variables, with potential applications ranging from attribution of responsibility in legal or AI systems, to the analysis of biological networks or climate models.

Method: The study introduces a framework to assess sycophantic behavior using AMPS (mathematics) and MedQuad (medical advice) datasets, testing three major LLMs with different rebuttal strategies (preemptive vs. in-context, simple vs. citation-based).

Result: Sycophantic behavior occurred in 58.19% of cases, with Gemini showing highest rates (62.47%). Progressive sycophancy (leading to correct answers) was 43.52%, while regressive sycophancy (leading to incorrect answers) was 14.66%. Preemptive rebuttals showed higher sycophancy than in-context rebuttals (61.75% vs. 56.52%).

Conclusion: Sycophantic behavior is highly persistent (78.5%) across models and contexts, highlighting both risks and opportunities for LLM deployment in structured domains, with implications for prompt programming and model optimization for safer AI applications.

Abstract: Large language models (LLMs) are increasingly applied in educational, clinical, and professional settings, but their tendency for sycophancy – prioritizing user agreement over independent reasoning – poses risks to reliability. This study introduces a framework to evaluate sycophantic behavior in ChatGPT-4o, Claude-Sonnet, and Gemini-1.5-Pro across AMPS (mathematics) and MedQuad (medical advice) datasets. Sycophantic behavior was observed in 58.19% of cases, with Gemini exhibiting the highest rate (62.47%) and ChatGPT the lowest (56.71%). Progressive sycophancy, leading to correct answers, occurred in 43.52% of cases, while regressive sycophancy, leading to incorrect answers, was observed in 14.66%. Preemptive rebuttals demonstrated significantly higher sycophancy rates than in-context rebuttals (61.75% vs. 56.52%, $Z=5.87$, $p<0.001$), particularly in computational tasks, where regressive sycophancy increased significantly (preemptive: 8.13%, in-context: 3.54%, $p<0.001$). Simple rebuttals maximized progressive sycophancy ($Z=6.59$, $p<0.001$), while citation-based rebuttals exhibited the highest regressive rates ($Z=6.59$, $p<0.001$). Sycophantic behavior showed high persistence (78.5%, 95% CI: [77.2%, 79.8%]) regardless of context or model. These findings emphasize the risks and opportunities of deploying LLMs in structured and dynamic domains, offering insights into prompt programming and model optimization for safer AI applications.

Method: Using the HEXACO personality framework from social psychology, researchers designed experimental prompts to test three LLMs’ performance on three toxic and bias benchmarks.

Result: All three models showed sensitivity to HEXACO personality traits, with consistent variations in biases, negative sentiment, and toxicity. Adjusting specific personality trait levels effectively reduced bias and toxicity.

Conclusion: The findings emphasize the need to examine content safety beyond training efficiency for LLM personification and suggest personality adjustment as a simple, low-cost method for controlled text generation.

Abstract: With the different roles that AI is expected to play in human life, imbuing large language models (LLMs) with different personalities has attracted increasing research interests. While the “personification” enhances human experiences of interactivity and adaptability of LLMs, it gives rise to critical concerns about content safety, particularly regarding bias, sentiment and toxicity of LLM generation. This study explores how assigning different personality traits to LLMs affects the toxicity and biases of their outputs. Leveraging the widely accepted HEXACO personality framework developed in social psychology, we design experimentally sound prompts to test three LLMs’ performance on three toxic and bias benchmarks. The findings demonstrate the sensitivity of all three models to HEXACO personality traits and, more importantly, a consistent variation in the biases, negative sentiment and toxicity of their output. In particular, adjusting the levels of several personality traits can effectively reduce bias and toxicity in model performance, similar to humans’ correlations between personality traits and toxic behaviors. The findings highlight the additional need to examine content safety besides the efficiency of training or fine-tuning methods for LLM personification. They also suggest a potential for the adjustment of personalities to be a simple and low-cost method to conduct controlled text generation.

Method: The approach transfers safety interventions between models through learned mappings of shared activation spaces, using steering vectors to alter models’ outputs predictably. It introduces a new ‘corrupted capabilities’ task where models are fine-tuned to embed knowledge tied to a backdoor.

Result: Extensive experiments across Llama, Qwen and Gemma model families show successful transfer of steering vectors for backdoor removal and refusal of harmful prompts. Autoencoder mappings between base and fine-tuned models serve as reliable ’lightweight safety switches’ for dynamic behavior toggling.

Conclusion: The method enables efficient alignment of larger models using smaller ones, with autoencoder mappings providing practical safety mechanisms that can dynamically toggle between model behaviors.

Abstract: The study of representation universality in AI models reveals growing convergence across domains, modalities, and architectures. However, the practical applications of representation universality remain largely unexplored. We bridge this gap by demonstrating that safety interventions can be transferred between models through learned mappings of their shared activation spaces. We demonstrate this approach on two well-established AI safety tasks: backdoor removal and refusal of harmful prompts, showing successful transfer of steering vectors that alter the models’ outputs in a predictable way. Additionally, we propose a new task, \textit{corrupted capabilities}, where models are fine-tuned to embed knowledge tied to a backdoor. This tests their ability to separate useful skills from backdoors, reflecting real-world challenges. Extensive experiments across Llama, Qwen and Gemma model families show that our method enables using smaller models to efficiently align larger ones. Furthermore, we demonstrate that autoencoder mappings between base and fine-tuned models can serve as reliable ``lightweight safety switches”, allowing dynamic toggling between model behaviors.

Method: Proposes DebFlow framework that employs debate mechanism to optimize workflows and integrates reflexion to improve based on previous experiences.

Result: Achieved 3% average performance improvement across six benchmark datasets (HotpotQA, MATH, ALFWorld) and reduced resource consumption by 37% during training compared to state-of-the-art baselines.

Conclusion: Debate component plays a critical role in enhancing framework performance (4% drop when removed), while reflexion provides auxiliary contribution (2% drop when removed), demonstrating the effectiveness of the proposed approach.

Abstract: Large language models (LLMs) have demonstrated strong potential and impressive performance in automating the generation and optimization of workflows. However, existing approaches are marked by limited reasoning capabilities, high computational demands, and significant resource requirements. To address these issues, we propose DebFlow, a framework that employs a debate mechanism to optimize workflows and integrates reflexion to improve based on previous experiences. We evaluated our method across six benchmark datasets, including HotpotQA, MATH, and ALFWorld. Our approach achieved a 3% average performance improvement over the latest baselines, demonstrating its effectiveness in diverse problem domains. In particular, during training, our framework reduces resource consumption by 37% compared to the state-of-the-art baselines. Additionally, we performed ablation studies. Removing the Debate component resulted in a 4% performance drop across two benchmark datasets, significantly greater than the 2% drop observed when the Reflection component was removed. These findings strongly demonstrate the critical role of Debate in enhancing framework performance, while also highlighting the auxiliary contribution of reflexion to overall optimization.

Method: Proposes a deployment-centric workflow that complements data-centric and model-centric approaches. Identifies common multimodal-AI-specific challenges across disciplines and examines three real-world use cases: pandemic response, self-driving car design, and climate change adaptation.

Result: The approach facilitates multidisciplinary dialogue and open research practices, enabling the community to accelerate deployment-centric development for broader societal impact.

Conclusion: By integrating deployment constraints early and fostering deeper multimodal integration across disciplines, multimodal AI can achieve more significant real-world impact beyond current vision/language limitations.

Abstract: Multimodal artificial intelligence (AI) integrates diverse types of data via machine learning to improve understanding, prediction, and decision-making across disciplines such as healthcare, science, and engineering. However, most multimodal AI advances focus on models for vision and language data, while their deployability remains a key challenge. We advocate a deployment-centric workflow that incorporates deployment constraints early to reduce the likelihood of undeployable solutions, complementing data-centric and model-centric approaches. We also emphasise deeper integration across multiple levels of multimodality and multidisciplinary collaboration to significantly broaden the research scope beyond vision and language. To facilitate this approach, we identify common multimodal-AI-specific challenges shared across disciplines and examine three real-world use cases: pandemic response, self-driving car design, and climate change adaptation, drawing expertise from healthcare, social science, engineering, science, sustainability, and finance. By fostering multidisciplinary dialogue and open research practices, our community can accelerate deployment-centric development for broad societal impact.

Method: Created SPaRC - a dataset of 1,000 2D grid pathfinding puzzles requiring step-by-step planning with arithmetic and geometric rules, then evaluated human performance and various reasoning models including o4-mini.

Result: Humans achieved near-perfect accuracy (98.0%), while best models like o4-mini struggled (15.8%, 1.1% on hard puzzles). Models frequently generated invalid paths (>50%) and showed navigation/spatial logic errors. Unlike humans, models failed to scale test-time compute with difficulty.

Conclusion: SPaRC reveals significant spatial reasoning limitations in current AI models and can drive research toward better methods for abstract, multi-step problem-solving, with potential improvements through better training and test-time scaling.

Abstract: Existing reasoning datasets saturate and fail to test abstract, multi-step problems, especially pathfinding and complex rule constraint satisfaction. We introduce SPaRC (Spatial Pathfinding Reasoning Challenge), a dataset of 1,000 2D grid pathfinding puzzles to evaluate spatial and symbolic reasoning, requiring step-by-step planning with arithmetic and geometric rules. Humans achieve near-perfect accuracy (98.0%; 94.5% on hard puzzles), while the best reasoning models, such as o4-mini, struggle (15.8%; 1.1% on hard puzzles). Models often generate invalid paths (>50% of puzzles for o4-mini), and reasoning tokens reveal they make errors in navigation and spatial logic. Unlike humans, who take longer on hard puzzles, models fail to scale test-time compute with difficulty. Allowing models to make multiple solution attempts improves accuracy, suggesting potential for better spatial reasoning with improved training and efficient test-time scaling methods. SPaRC can be used as a window into models’ spatial reasoning limitations and drive research toward new methods that excel in abstract, multi-step problem-solving.

Method: Developed a benchmark from real-world academic literature with diverse texts varying in length, complexity, and domains. Evaluated LLMs on this benchmark.

Result: LLMs face significant challenges in real-world causal inference, with the best model achieving only 0.477 F1 score. Analysis revealed difficulties with confounding, graph size, text length, and domain variations.

Conclusion: The benchmark provides targeted insights for advancing LLM causal reasoning, highlighting the gap between synthetic and real-world performance and the need for improved methods.

Abstract: Understanding and inferring causal relationships from texts is a core aspect of human cognition and is essential for advancing large language models (LLMs) towards artificial general intelligence. Existing work evaluating LLM causal reasoning primarily focuses on synthetically generated texts which involve straightforward causal relationships that are explicitly mentioned in the text. This fails to reflect the complexities of real-world tasks. In this paper, we investigate whether LLMs are capable of inferring causal relationships from real-world texts. We develop a benchmark drawn from real-world academic literature which includes diverse texts with respect to length, complexity of relationships (different levels of explicitness, number of nodes, and causal relationships), and domains and sub-domains. To the best of our knowledge, our benchmark is the first-ever real-world dataset for this task. Our experiments on this dataset show that LLMs face significant challenges in inferring causal relationships from real-world text, with the best-performing model achieving an average F1 score of only 0.477. Through systematic analysis across aspects of real-world text (degree of confounding, size of graph, length of text, domain), our benchmark offers targeted insights for further research into advancing LLM causal reasoning.

Method: Dynamics modelling (DyMo) augments LLMs with state prediction alongside function calling during post-training, enabling them to predict future states through an internal environment model. This is integrated with self-verification sampling (SVS).

Result: On the Berkeley Function Calling Leaderboard V2, DyMo improves success rates and significantly reduces hallucinations. Combined with SVS, it substantially improves pass^k over trials k and allows models to refuse unreliable outputs.

Conclusion: DyMo and SVS together enhance LLM effectiveness and reliability for tool use, charting a path towards scalable planning RL methods without repeatedly querying the oracle environment.

Abstract: Tool use in stateful environments presents unique challenges for large language models (LLMs), where existing test-time compute strategies relying on repeated trials in the environment are impractical. We propose dynamics modelling (DyMo), a method that augments LLMs with a state prediction capability alongside function calling during post-training. This enables LLMs to predict the future states of their actions through an internal environment model. On the Berkeley Function Calling Leaderboard V2, DyMo improves success rates and significantly reduces hallucinations. We further integrate the internal environment model into self-verification sampling (SVS), and show that this substantially improves pass^k over number of trials k, and allows the model to refuse unreliable outputs. Together, DyMo and SVS greatly enhance the effectiveness and reliability of LLMs for tool use. We believe this work charts a path towards scalable planning RL methods for LLM inference without repeatedly querying the oracle environment.

Method: The Reproducibility Copilot analyzes manuscripts, code, and supplementary materials to generate structured Jupyter Notebooks and recommendations, systematically detecting barriers like missing hyperparameters, undocumented preprocessing steps, and inaccessible datasets.

Result: Feasibility tests showed OpenPub can substantially reduce reproduction time from over 30 hours to about 1 hour while achieving high coverage of figures, tables, and results suitable for computational reproduction.

Conclusion: AI-driven tools like OpenPub can meaningfully reduce the burden of reproducibility efforts and contribute to more transparent science, with the modular architecture providing a foundation for extending AI assistance to additional open science objectives.

Abstract: Open science initiatives seek to make research outputs more transparent, accessible, and reusable, but ensuring that published findings can be independently reproduced remains a persistent challenge. This paper introduces OpenPub, an AI-powered platform that supports researchers, reviewers, and readers through a suite of modular copilots focused on key open science tasks. In this work, we present the Reproducibility Copilot, which analyzes manuscripts, code, and supplementary materials to generate structured Jupyter Notebooks and recommendations aimed at facilitating computational, or “rote”, reproducibility. We conducted feasibility tests using previously studied research papers with known reproducibility benchmarks. Results indicate that OpenPub can substantially reduce reproduction time - from over 30 hours to about 1 hour - while achieving high coverage of figures, tables, and results suitable for computational reproduction. The system systematically detects barriers to reproducibility, including missing hyperparameters, undocumented preprocessing steps, and incomplete or inaccessible datasets. While preliminary, these findings suggest that AI-driven tools can meaningfully reduce the burden of reproducibility efforts and contribute to more transparent and verifiable scientific communication. The modular copilot architecture also provides a foundation for extending AI assistance to additional open science objectives beyond reproducibility.

Method: Organize data in knowledge graphs using BFO and CCO ontologies, retrieve query results to create Markov chain models for prediction, introduce ‘spatiotemporal instant’ concept, and propose alternative probability model based on actual process profiles.

Result: Demonstrated successful prediction of fishing vessel movements using the proposed framework, with seamless integration of probability calculations back into the knowledge graph.

Conclusion: Ontology-structured knowledge graphs with proper semantic frameworks can effectively support future event prediction, and an alternative view of probability as being about actual process profiles better captures real-world dynamics.

Abstract: We argue that ontology-structured knowledge graphs can play a crucial role in generating predictions about future events. By leveraging the semantic framework provided by Basic Formal Ontology (BFO) and Common Core Ontologies (CCO), we demonstrate how data such as the movements of a fishing vessel can be organized in and retrieved from a knowledge graph. These query results are then used to create Markov chain models, allowing us to predict future states based on the vessel’s history. To fully support this process, we introduce the term `spatiotemporal instant’ to complete the necessary structural semantics. Additionally, we critique the prevailing ontological model of probability, according to which probabilities are about the future. We propose an alternative view, where at least some probabilities are treated as being about actual process profiles, which better captures the dynamics of real-world phenomena. Finally, we demonstrate how our Markov chain-based probability calculations can be seamlessly integrated back into the knowledge graph, enabling further analysis and decision-making.

Method: Modular configuration-based pipeline with dual-stage quality tagging (heuristic rules + LLM evaluations) to filter OASST-formatted conversations.

Result: Produces structured datasets supporting both SFT and DPO use cases with high-fidelity dialogue samples.

Conclusion: The framework provides a robust solution for scalable synthetic data generation, significantly reducing data preparation costs in LLM training.

Abstract: The advancement of large language models (LLMs) is critically dependent on the availability of high-quality datasets for Supervised Fine-Tuning (SFT), alignment tasks like Direct Preference Optimization (DPO), etc. In this work, we present a comprehensive synthetic data generation framework that facilitates scalable, configurable, and high-fidelity generation of synthetic data tailored for these training paradigms. Our approach employs a modular and configuration-based pipeline capable of modeling complex dialogue flows with minimal manual intervention. This framework uses a dual-stage quality tagging mechanism, combining heuristic rules and LLM-based evaluations, to automatically filter and score data extracted from OASST-formatted conversations, ensuring the curation of high-quality dialogue samples. The resulting datasets are structured under a flexible schema supporting both SFT and DPO use cases, enabling seamless integration into diverse training workflows. Together, these innovations offer a robust solution for generating and managing synthetic conversational data at scale, significantly reducing the overhead of data preparation in LLM training pipelines.

Method: Proposes a training-free framework with Adaptive Planning Graph consisting of planning, retrieval and reasoning modules. The planning module dynamically determines next actions and graph expansion. Uses modality-specific strategies for flexible retrieval across different data types.

Result: Experiments on MultimodalQA and WebQA show the approach matches or outperforms existing models that require training, demonstrating effective multimodal reasoning without task-specific training.

Conclusion: The training-free framework enables dynamic and flexible exploration of reasoning paths while preserving multimodal information characteristics, providing a cost-effective alternative to traditional trained models.

Abstract: Multimodal Multi-hop question answering requires integrating information from diverse sources, such as images and texts, to derive answers. Existing methods typically rely on sequential retrieval and reasoning, where each step builds on the previous output. However, this single-path paradigm makes them vulnerable to errors due to misleading intermediate steps. Moreover, developing multimodal models can be computationally expensive, often requiring extensive training. To address these limitations, we propose a training-free framework guided by an Adaptive Planning Graph, which consists of planning, retrieval and reasoning modules. The planning module analyzes the current state of the Adaptive Planning Graph, determines the next action and where to expand the graph, which enables dynamic and flexible exploration of reasoning paths. To handle retrieval of text to unspecified target modalities, we devise modality-specific strategies that dynamically adapt to distinct data types. Our approach preserves the characteristics of multimodal information without costly task-specific training, enabling seamless integration with up-to-date models. Finally, the experiments on MultimodalQA and WebQA show that our approach matches or outperforms existing models that rely on training.

Method: Compiled 13 latest Olympiad exams (2024-2025) with mixed modalities, adopted official marking schemes for fine-grained grading, and assigned medals based on official thresholds to compare models with human contestants.

Result: Evaluation of 30 state-of-the-art (M)LLMs shows: open-source MLLMs mostly at/below bronze level; open-source LLMs show promising progress with multiple golds; closed-source reasoning MLLMs achieve 6-12 gold medals; most models still have significant gap from full marks.

Conclusion: HiPhO reveals performance gaps between open-source models and top students, demonstrates strong reasoning abilities of closed-source models, and highlights remaining room for improvement in multimodal physical reasoning.

Abstract: Recently, the physical capabilities of (M)LLMs have garnered increasing attention. However, existing benchmarks for physics suffer from two major gaps: they neither provide systematic and up-to-date coverage of real-world physics competitions such as physics Olympiads, nor enable direct performance comparison with humans. To bridge these gaps, we present HiPhO, the first benchmark dedicated to high school physics Olympiads with human-aligned evaluation. Specifically, HiPhO highlights three key innovations. (1) Comprehensive Data: It compiles 13 latest Olympiad exams from 2024-2025, spanning both international and regional competitions, and covering mixed modalities that encompass problems spanning text-only to diagram-based. (2) Professional Evaluation: We adopt official marking schemes to perform fine-grained grading at both the answer and step level, fully aligned with human examiners to ensure high-quality and domain-specific evaluation. (3) Comparison with Human Contestants: We assign gold, silver, and bronze medals to models based on official medal thresholds, thereby enabling direct comparison between (M)LLMs and human contestants. Our large-scale evaluation of 30 state-of-the-art (M)LLMs shows that: across 13 exams, open-source MLLMs mostly remain at or below the bronze level; open-source LLMs show promising progress with multiple golds; closed-source reasoning MLLMs can achieve 6 to 12 gold medals; and most models still have a significant gap from full marks. These results highlight the performance gap between open-source models and top students, the strong reasoning abilities of closed-source models, and the remaining room for improvement. HiPhO, a human-aligned Olympiad benchmark for multimodal physical reasoning, is open-source at https://github.com/SciYu/HiPhO with a public leaderboard at https://phyarena.github.io/.

Method: Robust Sparse Sampling (RSS) computes robust value functions by leveraging Sample Average Approximation (SAA), making it tractable for online settings. It’s applicable to infinite/continuous state spaces with sample and computational complexities independent of state space size.

Result: RSS outperforms standard Sparse Sampling in environments with uncertain dynamics, providing theoretical performance guarantees while maintaining computational efficiency suitable for real-time applications.

Conclusion: RSS successfully bridges the gap between robust planning theory and practical online applications, offering a computationally feasible solution for robust decision-making under model uncertainty with provable guarantees.

Abstract: Online planning in Markov Decision Processes (MDPs) enables agents to make sequential decisions by simulating future trajectories from the current state, making it well-suited for large-scale or dynamic environments. Sample-based methods such as Sparse Sampling and Monte Carlo Tree Search (MCTS) are widely adopted for their ability to approximate optimal actions using a generative model. However, in practical settings, the generative model is often learned from limited data, introducing approximation errors that can degrade performance or lead to unsafe behaviors. To address these challenges, Robust MDPs (RMDPs) offer a principled framework for planning under model uncertainty, yet existing approaches are typically computationally intensive and not suited for real-time use. In this work, we introduce Robust Sparse Sampling (RSS), the first online planning algorithm for RMDPs with finite-sample theoretical performance guarantees. Unlike Sparse Sampling, which estimates the nominal value function, RSS computes a robust value function by leveraging the efficiency and theoretical properties of Sample Average Approximation (SAA), enabling tractable robust policy computation in online settings. RSS is applicable to infinite or continuous state spaces, and its sample and computational complexities are independent of the state space size. We provide theoretical performance guarantees and empirically show that RSS outperforms standard Sparse Sampling in environments with uncertain dynamics.

Method: Analyzed vision-language descriptions from NVIDIA’s Describe Anything Model evaluated by three GPT variants, with controlled experiments using Gemini 2.5 Pro as independent question generator and cross-family semantic similarity analysis.

Result: GPT models cluster together with high similarity while Gemini shows markedly different evaluation strategies. All GPT models consistently favor negative assessment over positive confirmation by 2:1 ratio.

Conclusion: Evaluation competence doesn’t scale with general AI capability, and robust AI assessment requires diverse architectural perspectives to mitigate inherent biases.

Abstract: As AI systems increasingly evaluate other AI outputs, understanding their assessment behavior becomes crucial for preventing cascading biases. This study analyzes vision-language descriptions generated by NVIDIA’s Describe Anything Model and evaluated by three GPT variants (GPT-4o, GPT-4o-mini, GPT-5) to uncover distinct “evaluation personalities” the underlying assessment strategies and biases each model demonstrates. GPT-4o-mini exhibits systematic consistency with minimal variance, GPT-4o excels at error detection, while GPT-5 shows extreme conservatism with high variability. Controlled experiments using Gemini 2.5 Pro as an independent question generator validate that these personalities are inherent model properties rather than artifacts. Cross-family analysis through semantic similarity of generated questions reveals significant divergence: GPT models cluster together with high similarity while Gemini exhibits markedly different evaluation strategies. All GPT models demonstrate a consistent 2:1 bias favoring negative assessment over positive confirmation, though this pattern appears family-specific rather than universal across AI architectures. These findings suggest that evaluation competence does not scale with general capability and that robust AI assessment requires diverse architectural perspectives.

Method: Uses image processing for character detection, text recognition, and emotion intensity recognition; LLM for dialogue attribution and emotion analysis; TTS with character-specific voice profiles.

Result: Developed a complete pipeline that takes comic volumes as input and produces speech aligned with character dialogue and emotional states.

Conclusion: This work represents a step toward interactive comic reading by automating voiceover generation with character and emotion awareness.

Abstract: This paper presents an end-to-end pipeline for generating character-specific, emotion-aware speech from comics. The proposed system takes full comic volumes as input and produces speech aligned with each character’s dialogue and emotional state. An image processing module performs character detection, text recognition, and emotion intensity recognition. A large language model performs dialogue attribution and emotion analysis by integrating visual information with the evolving plot context. Speech is synthesized through a text-to-speech model with distinct voice profiles tailored to each character and emotion. This work enables automated voiceover generation for comics, offering a step toward interactive and immersive comic reading experience.

Method: Systematically examined three fine-tuning schemes: text-only, direct mixing, and curriculum learning for spoken language understanding (SLU), focusing on scenarios with abundant text-label pairs but limited speech-label data.

Result: LALMs achieve competitive performance with text-only fine-tuning. Adding 2-5% speech data yields substantial gains. Curriculum learning is particularly effective under scarce data. Cross-lingual SLU adaptation works well by combining source-language speech with target-language text and minimal target-language speech.

Conclusion: This study provides practical insights into LALM fine-tuning under realistic data constraints, showing that text-only fine-tuning is surprisingly effective and small amounts of speech data can significantly boost performance.

Abstract: Large Audio Language Models (LALMs) have emerged as powerful tools for speech-related tasks but remain underexplored for fine-tuning, especially with limited speech data. To bridge this gap, we systematically examine how different fine-tuning schemes including text-only, direct mixing, and curriculum learning affect spoken language understanding (SLU), focusing on scenarios where text-label pairs are abundant while paired speech-label data are limited. Results show that LALMs already achieve competitive performance with text-only fine-tuning, highlighting their strong generalization ability. Adding even small amounts of speech data (2-5%) yields substantial further gains, with curriculum learning particularly effective under scarce data. In cross-lingual SLU, combining source-language speech data with target-language text and minimal target-language speech data enables effective adaptation. Overall, this study provides practical insights into the LALM fine-tuning under realistic data constraints.

Method: Using DeepSpeech as ASR target, generated targeted adversarial examples and evaluated their impact on speaker embeddings across genuine and impostor samples for 16 phonetically diverse target phrases.

Result: Adversarial perturbations exploit systematic confusions like vowel centralization and consonant substitutions, causing both transcription errors and identity drift that degrades phonetic cues critical for speaker verification.

Conclusion: Phonetic-aware defenses are needed to ensure robustness of ASR and speaker recognition systems against adversarial attacks that simultaneously affect transcription accuracy and speaker identity.

Abstract: Adversarial perturbations in speech pose a serious threat to automatic speech recognition (ASR) and speaker verification by introducing subtle waveform modifications that remain imperceptible to humans but can significantly alter system outputs. While targeted attacks on end-to-end ASR models have been widely studied, the phonetic basis of these perturbations and their effect on speaker identity remain underexplored. In this work, we analyze adversarial audio at the phonetic level and show that perturbations exploit systematic confusions such as vowel centralization and consonant substitutions. These distortions not only mislead transcription but also degrade phonetic cues critical for speaker verification, leading to identity drift. Using DeepSpeech as our ASR target, we generate targeted adversarial examples and evaluate their impact on speaker embeddings across genuine and impostor samples. Results across 16 phonetically diverse target phrases demonstrate that adversarial audio induces both transcription errors and identity drift, highlighting the need for phonetic-aware defenses to ensure the robustness of ASR and speaker recognition systems.

Method: The approach uses generative voice models to decompose audio signals into high-level semantic representations of fundamentally distinct features, enabling a semantic communications framework that prioritizes meaningful content over uniform feature representation.

Result: The technique matches or outperforms existing audio codecs on transcription, sentiment analysis, and speaker verification at 2-4x lower bitrates. It notably surpasses Encodec in perceptual quality and speaker verification while using up to 4x less bitrate.

Conclusion: Semantic communications leveraging generative voice models’ factorized representations can achieve significantly lower bitrates while maintaining or improving perceptual quality and downstream task performance compared to traditional audio codecs.

Abstract: While existing speech audio codecs designed for compression exploit limited forms of temporal redundancy and allow for multi-scale representations, they tend to represent all features of audio in the same way. In contrast, generative voice models designed for text-to-speech and voice transfer tasks have recently proved effective at factorizing audio signals into high-level semantic representations of fundamentally distinct features. In this paper, we leverage such representations in a novel semantic communications approach to achieve lower bitrates without sacrificing perceptual quality or suitability for specific downstream tasks. Our technique matches or outperforms existing audio codecs on transcription, sentiment analysis, and speaker verification when encoding at 2-4x lower bitrate – notably surpassing Encodec in perceptual quality and speaker verification while using up to 4x less bitrate.

Method: Two-stage modeling: (1) video-guided audio semantic model predicts unified audio semantic tokens using phonetic cues; (2) video-conditioned semantic-to-acoustic model refines semantic tokens into detailed acoustic tokens.

Result: BVS effectively generates synchronized audio with context-aware speech for videos, demonstrated in video-to-context-aware speech synthesis and immersive audio background conversion scenarios.

Conclusion: The proposed BVS method successfully addresses the gap in generating environmentally aware intelligible speech synchronized with video content, with ablation studies validating the design.

Abstract: The generation of realistic, context-aware audio is important in real-world applications such as video game development. While existing video-to-audio (V2A) methods mainly focus on Foley sound generation, they struggle to produce intelligible speech. Meanwhile, current environmental speech synthesis approaches remain text-driven and fail to temporally align with dynamic video content. In this paper, we propose Beyond Video-to-SFX (BVS), a method to generate synchronized audio with environmentally aware intelligible speech for given videos. We introduce a two-stage modeling method: (1) stage one is a video-guided audio semantic (V2AS) model to predict unified audio semantic tokens conditioned on phonetic cues; (2) stage two is a video-conditioned semantic-to-acoustic (VS2A) model that refines semantic tokens into detailed acoustic tokens. Experiments demonstrate the effectiveness of BVS in scenarios such as video-to-context-aware speech synthesis and immersive audio background conversion, with ablation studies further validating our design. Our demonstration is available at~\href{https://xinleiniu.github.io/BVS-demo/}{BVS-Demo}.

Method: A data augmentation method that emphasizes high-frequency information in contrastive learning, enabling the model to pay more attention to low-frequency information of normal machine audio.

Result: Outperformed other contrastive learning methods on DCASE 2020 Task 2 dataset and demonstrated good generalizability on DCASE 2022 Task 2 dataset.

Conclusion: The proposed high-frequency data augmentation method effectively improves anomaly sound detection performance by directing model attention to low-frequency normal operational sounds.

Abstract: The outlier exposure method is an effective approach to address the unsupervised anomaly sound detection problem. The key focus of this method is how to make the model learn the distribution space of normal data. Based on biological perception and data analysis, it is found that anomalous audio and noise often have higher frequencies. Therefore, we propose a data augmentation method for high-frequency information in contrastive learning. This enables the model to pay more attention to the low-frequency information of the audio, which represents the normal operational mode of the machine. We evaluated the proposed method on the DCASE 2020 Task 2. The results showed that our method outperformed other contrastive learning methods used on this dataset. We also evaluated the generalizability of our method on the DCASE 2022 Task 2 dataset.

Method: The framework involves: 1) constructing a novel CoT dataset for TS-ASR, 2) training the TS-ASR model on regular data followed by fine-tuning on CoT data, and 3) further training with RL using selected data to enhance generalized reasoning capabilities.

Result: Experimental results demonstrate significant improvement in TS-ASR performance with CoT and RL training, establishing state-of-the-art performance compared to previous TS-ASR works on comparable datasets.

Conclusion: The integration of CoT and RL training effectively enhances TS-ASR capabilities, providing a powerful approach for handling complex multi-speaker scenarios through improved reasoning and generalization.

Abstract: Target Speaker Automatic Speech Recognition (TS-ASR) aims to transcribe the speech of a specified target speaker from multi-speaker mixtures in cocktail party scenarios. Recent advancement of Large Audio-Language Models (LALMs) has already brought some new insights to TS-ASR. However, significant room for optimization remains for the TS-ASR task within the LALMs architecture. While Chain of Thoughts (CoT) and Reinforcement Learning (RL) have proven effective in certain speech tasks, TS-ASR, which requires the model to deeply comprehend speech signals, differentiate various speakers, and handle overlapping utterances is particularly well-suited to a reasoning-guided approach. Therefore, we propose a novel framework that incorporates CoT and RL training into TS-ASR for performance improvement. A novel CoT dataset of TS-ASR is constructed, and the TS-ASR model is first trained on regular data and then fine-tuned on CoT data. Finally, the model is further trained with RL using selected data to enhance generalized reasoning capabilities. Experiment results demonstrate a significant improvement of TS-ASR performance with CoT and RL training, establishing a state-of-the-art performance compared with previous works of TS-ASR on comparable datasets.

Method: The authors derive constraints for asymptotically stable variants of commonly used recurrent neural networks and test their approach under zero input and time-varied conditioning scenarios.

Result: Asymptotic stability in recurrent neural networks successfully eliminates audio artifacts from model output under time-varied control conditioning.

Conclusion: The findings suggest a general solution for mitigating conditioning-induced artifacts in various audio neural network architectures including convolutional and state-space models.

Abstract: Recurrent neural networks are used in virtual analog modeling applications to digitally replicate the sound of analog hardware audio processors. The controls of hardware devices can be used as a conditioning input to these networks. A common method for introducing control conditioning to these models is the direct static concatenation of controls with input audio samples, which we show produces audio artifacts under time-varied conditioning. Here we derive constraints for asymptotically stable variants of commonly used recurrent neural networks and demonstrate that asymptotical stability in recurrent neural networks can eliminate audio artifacts from the model output under zero input and time-varied conditioning. Furthermore, our results suggest a possible general solution to mitigate conditioning-induced artifacts in other audio neural network architectures, such as convolutional and state-space models.

Method: A masked transformer model that takes two audio prompts: one for the desired rhythmic pattern and another for drumkit timbre, then generates drum audio with appropriate elaborations.

Result: The model trained on less than 10 hours of publicly-available drum data can generate high-quality, faithful realizations of sound gestures across various timbres in a zero-shot manner, as shown by subjective and objective evaluations.

Conclusion: TRIA successfully enables efficient conversion of rhythmic sound gestures to professional drum recordings, demonstrating the effectiveness of masked transformer models for this audio generation task with limited training data.

Abstract: Musicians and nonmusicians alike use rhythmic sound gestures, such as tapping and beatboxing, to express drum patterns. While these gestures effectively communicate musical ideas, realizing these ideas as fully-produced drum recordings can be time-consuming, potentially disrupting many creative workflows. To bridge this gap, we present TRIA (The Rhythm In Anything), a masked transformer model for mapping rhythmic sound gestures to high-fidelity drum recordings. Given an audio prompt of the desired rhythmic pattern and a second prompt to represent drumkit timbre, TRIA produces audio of a drumkit playing the desired rhythm (with appropriate elaborations) in the desired timbre. Subjective and objective evaluations show that a TRIA model trained on less than 10 hours of publicly-available drum data can generate high-quality, faithful realizations of sound gestures across a wide range of timbres in a zero-shot manner.

Method: Two proposed methods: 1) Training strategy using separate utterances for speaker identity and target impression, 2) Reference-free model that generates speaker embeddings solely from target impressions to improve robustness against leakage.

Result: Significant improvement in controllability with mean squared error reduction from 0.61 to 0.41 objectively and from 1.15 to 0.92 subjectively while maintaining high fidelity. Introduction of LibriTTS-VI, the first public voice impression dataset.

Conclusion: The proposed methods effectively mitigate impression leakage and improve voice impression controllability, with the reference-free model offering both improved robustness and convenience of reference-free generation.

Abstract: Fine-grained control over voice impressions (e.g., making a voice brighter or calmer) is a key frontier for creating more controllable text-to-speech. However, this nascent field faces two key challenges. The first is the problem of impression leakage, where the synthesized voice is undesirably influenced by the speaker’s reference audio, rather than the separately specified target impression, and the second is the lack of a public, annotated corpus. To mitigate impression leakage, we propose two methods: 1) a training strategy that separately uses an utterance for speaker identity and another utterance of the same speaker for target impression, and 2) a novel reference-free model that generates a speaker embedding solely from the target impression, achieving the benefits of improved robustness against the leakage and the convenience of reference-free generation. Objective and subjective evaluations demonstrate a significant improvement in controllability. Our best method reduced the mean squared error of 11-dimensional voice impression vectors from 0.61 to 0.41 objectively and from 1.15 to 0.92 subjectively, while maintaining high fidelity. To foster reproducible research, we introduce LibriTTS-VI, the first public voice impression dataset released with clear annotation standards, built upon the LibriTTS-R corpus.

Method: Developed a new challenge database, introduced two tasks (singer identity and singing style conversion), open-sourced baselines, and conducted large-scale crowd-sourced listening tests and objective evaluations over two months.

Result: Top systems achieved comparable singer identity scores to ground truth samples, but modeling singing style and achieving high naturalness remained challenging, particularly for dynamic elements like breathy, glissando, and vibrato singing styles.

Conclusion: While voice conversion systems can effectively convert singer identity, modeling singing style and achieving naturalness remains difficult due to the complexity of dynamic vocal characteristics, indicating an area for future improvement.

Abstract: We present the findings of the latest iteration of the Singing Voice Conversion Challenge, a scientific event aiming to compare and understand different voice conversion systems in a controlled environment. Compared to previous iterations which solely focused on converting the singer identity, this year we also focused on converting the singing style of the singer. To create a controlled environment and thorough evaluations, we developed a new challenge database, introduced two tasks, open-sourced baselines, and conducted large-scale crowd-sourced listening tests and objective evaluations. The challenge was ran for two months and in total we evaluated 26 different systems. The results of the large-scale crowd-sourced listening test showed that top systems had comparable singer identity scores to ground truth samples. However, modeling the singing style and consequently achieving high naturalness still remains a challenge in this task, primarily due to the difficulty in modeling dynamic information in breathy, glissando, and vibrato singing styles.

Method: The framework incorporates reinforcement learning with two key innovations: Emotion Similarity-Weighted Reward (ESWR) and Explicit Structured Reasoning (ESR), built upon pretrained LALMs using group-relative policy optimization with emotion constraints.

Result: EMO-RL achieves state-of-the-art results on MELD and IEMOCAP datasets, with cross-dataset experiments demonstrating strong generalization superiority.

Conclusion: The proposed EMO-RL training strategies significantly enhance emotional reasoning capabilities of LALMs, overcoming limitations of ambiguous emotional boundaries and limited reasoning in smaller models.

Abstract: Although Large Audio-Language Models (LALMs) have exhibited outstanding performance in auditory understanding, their performance in affective computing scenarios, particularly in emotion recognition, reasoning, and subtle sentiment differentiation, remains suboptimal. Recent advances in Reinforcement Learning (RL) have shown promise in improving LALMs’ reasoning abilities. However, two critical challenges hinder the direct application of RL techniques to Speech Emotion Recognition (SER) tasks: (1) convergence instability caused by ambiguous emotional boundaries and (2) limited reasoning ability when using relatively small models (e.g., 7B-parameter architectures). To overcome these limitations, we introduce EMO-RL, a novel framework incorporating reinforcement learning with two key innovations: Emotion Similarity-Weighted Reward (ESWR) and Explicit Structured Reasoning (ESR). Built upon pretrained LALMs, our method employs group-relative policy optimization with emotion constraints. Comprehensive experiments demonstrate that our EMO-RL training strategies can significantly enhance the emotional reasoning capabilities of LALMs, attaining state-of-the-art results on both the MELD and IEMOCAP datasets, and cross-dataset experiments prove the strong superiority of generalization.

Method: Three-step framework: (i) test-time scaling to generate audio-focused chains of thought from LVLM teacher, (ii) audio-grounded validation to filter hallucinations, (iii) distillation pipeline with SFT followed by GRPO for LALM student.

Result: Improves LALM reasoning performance in both in-domain AVQA test set and unseen auditory scenes/questions, outperforming pretrained and label-only distilled baselines.

Conclusion: Vision reasoning can be effectively transferred to audio models and scaled with abundant audio-visual data.

Abstract: While large audio-language models (LALMs) have demonstrated state-of-the-art audio understanding, their reasoning capability in complex soundscapes still falls behind large vision-language models (LVLMs). Compared to the visual domain, one bottleneck is the lack of large-scale chain-of-thought audio data to teach LALM stepwise reasoning. To circumvent this data and modality gap, we present SightSound-R1, a cross-modal distillation framework that transfers advanced reasoning from a stronger LVLM teacher to a weaker LALM student on the same audio-visual question answering (AVQA) dataset. SightSound-R1 consists of three core steps: (i) test-time scaling to generate audio-focused chains of thought (CoT) from an LVLM teacher, (ii) audio-grounded validation to filter hallucinations, and (iii) a distillation pipeline with supervised fine-tuning (SFT) followed by Group Relative Policy Optimization (GRPO) for the LALM student. Results show that SightSound-R1 improves LALM reasoning performance both in the in-domain AVQA test set as well as in unseen auditory scenes and questions, outperforming both pretrained and label-only distilled baselines. Thus, we conclude that vision reasoning can be effectively transferred to audio models and scaled with abundant audio-visual data.

Method: Proposes TISDiSS framework with three key components: early-split multi-loss supervision, shared-parameter design, and dynamic inference repetitions. This enables adjusting inference depth for speed-performance trade-offs without retraining.

Result: Achieves state-of-the-art performance on standard speech separation benchmarks with reduced parameter count. Training with more inference repetitions improves shallow-inference performance for low-latency applications.

Conclusion: TISDiSS establishes a scalable and practical framework for adaptive source separation that provides flexible speed-performance trade-offs while maintaining high performance with fewer parameters.

Abstract: Source separation is a fundamental task in speech, music, and audio processing, and it also provides cleaner and larger data for training generative models. However, improving separation performance in practice often depends on increasingly large networks, inflating training and deployment costs. Motivated by recent advances in inference-time scaling for generative modeling, we propose Training-Time and Inference-Time Scalable Discriminative Source Separation (TISDiSS), a unified framework that integrates early-split multi-loss supervision, shared-parameter design, and dynamic inference repetitions. TISDiSS enables flexible speed-performance trade-offs by adjusting inference depth without retraining additional models. We further provide systematic analyses of architectural and training choices and show that training with more inference repetitions improves shallow-inference performance, benefiting low-latency applications. Experiments on standard speech separation benchmarks demonstrate state-of-the-art performance with a reduced parameter count, establishing TISDiSS as a scalable and practical framework for adaptive source separation.

Method: Built on Mamba-2 state-space model backbone, systematically explored design parameters including LLM sizes, LoRA ranks, and connector designs. Conducted in-depth analysis of LLM parameters, audio encoder fine-tuning strategies, audio feature diversity, and feature reduction/expansion techniques.

Result: Achieved strong captioning performance across benchmarks compared to larger language models trained on the same dataset, despite using fewer parameters.

Conclusion: The Mamba-2-based audio captioning model demonstrates efficient and effective performance, with comprehensive analysis providing insights into optimal design choices for audio-language models.

Abstract: We present an audio captioning model built on the Mamba-2 large language model backbone, which is a state-of-the-art (SOTA) state-space model (SSM). We systematically explore the design space: LLM sizes, LoRA ranks, and connector designs leveraging Mamba-2’s linear-time complexity with respect to sequence length. Across benchmarks, our models achieve strong captioning performance compared with larger language models trained on the same dataset, despite using fewer parameters. For the first time, we conduct an in-depth analysis of how the number of LLM parameters, audio encoder fine-tuning strategies, audio feature diversity, and different feature reduction or expansion techniques affect performance.

Method: Simultaneous Self-Augmentation (SimulSA) randomly truncates speech and constructs partially aligned translations, incorporating this simultaneous data into offline supervised fine-tuning data.

Result: Experimental results show that augmenting only about 1% of simultaneous data significantly activates LALMs’ Simul-S2TT capabilities without model architecture or decoding strategy changes.

Conclusion: SimulSA effectively bridges the distribution gap and enables simultaneous translation capabilities in large audio-language models with minimal data augmentation.

Abstract: Simultaneous speech-to-text translation (Simul-S2TT) aims to translate speech into target text in real time, outputting translations while receiving source speech input, rather than waiting for the entire utterance to be spoken. Simul-S2TT research often modifies model architectures to implement read-write strategies. However, with the rise of large audio-language models (LALMs), a key challenge is how to directly activate Simul-S2TT capabilities in base models without additional architectural changes. In this paper, we introduce {\bf Simul}taneous {\bf S}elf-{\bf A}ugmentation ({\bf SimulSA}), a strategy that utilizes LALMs’ inherent capabilities to obtain simultaneous data by randomly truncating speech and constructing partially aligned translation. By incorporating them into offline SFT data, SimulSA effectively bridges the distribution gap between offline translation during pretraining and simultaneous translation during inference. Experimental results demonstrate that augmenting only about {\bf 1%} of the simultaneous data, compared to the full offline SFT data, can significantly activate LALMs’ Simul-S2TT capabilities without modifications to model architecture or decoding strategy.

Method: Built upon BEATs, SONAR addresses three key challenges: joint sampling of new and prior data, regularization to balance specificity and generality, and dynamic expansion of the tokenizer codebook for novel acoustic patterns.

Result: Experiments across four distinct domains show that SONAR achieves high adaptability to new domains while maintaining robust resistance to catastrophic forgetting.

Conclusion: SONAR provides an efficient continual pre-training framework that enables audio representation models to adapt to new domains without expensive retraining, effectively balancing domain adaptation with knowledge preservation.

Abstract: Self-supervised learning (SSL) on large-scale datasets like AudioSet has become the dominant paradigm for audio representation learning. While the continuous influx of new, unlabeled audio presents an opportunity to enrich these static representations, a naive approach is to retrain the model from scratch using all available data. However, this method is computationally prohibitive and discards the valuable knowledge embedded in the previously trained model weights. To address this inefficiency, we propose SONAR (Self-distilled cONtinual pre-training for domain adaptive Audio Representation), a continual pre-training framework built upon BEATs. SONAR effectively adapts to new domains while mitigating catastrophic forgetting by tackling three key challenges: implementing a joint sampling strategy for new and prior data, applying regularization to balance specificity and generality, and dynamically expanding the tokenizer codebook for novel acoustic patterns. Experiments across four distinct domains demonstrate that our method achieves both high adaptability and robust resistance to forgetting.

Method: Proposes EmoQ framework with EmoQ-Former for generating query embeddings that fuse multimodal information, multi-objective affective learning (MAL) for co-optimization, and soft-prompt injection strategy to inject multimodal representations into the LLM.

Result: Achieves state-of-the-art performance on IEMOCAP and MELD datasets, demonstrating superior emotion recognition capabilities.

Conclusion: EmoQ provides a new multimodal fusion paradigm for SER that effectively addresses current limitations and achieves superior performance through end-to-end architecture.

Abstract: The performance of speech emotion recognition (SER) is limited by the insufficient emotion information in unimodal systems and the feature alignment difficulties in multimodal systems. Recently, multimodal large language models (MLLMs) have made progress in SER. However, MLLMs still suffer from hallucination and misclassification problems in complex emotion reasoning. To address these problems, we propose an MLLM-based framework called EmoQ, which generates query embeddings that fuse multimodal information through an EmoQ-Former and uses multi-objective affective learning (MAL) to achieve co-optimization. The framework also provides a soft-prompt injection strategy to inject multimodal representations into the LLM. This end-to-end architecture achieves state-of-the-art performance on the IEMOCAP and MELD datasets, providing a new multimodal fusion paradigm for SER.

Method: Uses speaker-specific deviation distributions for intra-speaker temporal consistency, Markov chain for turn-taking, fixed room impulse response for spatial realism, and unified pause/overlap gap distribution modeled with kernel density estimation.

Result: Evaluation on Switchboard shows speaker-aware simulation better aligns with real conversational patterns than baseline, capturing fine-grained temporal dependencies and realistic speaker alternation.

Conclusion: The speaker-aware approach effectively models conversational dynamics but reveals challenges in modeling long-range conversational structure.

Abstract: We present a speaker-aware approach for simulating multi-speaker conversations that captures temporal consistency and realistic turn-taking dynamics. Prior work typically models aggregate conversational statistics under an independence assumption across speakers and turns. In contrast, our method uses speaker-specific deviation distributions enforcing intra-speaker temporal consistency, while a Markov chain governs turn-taking and a fixed room impulse response preserves spatial realism. We also unify pauses and overlaps into a single gap distribution, modeled with kernel density estimation for smooth continuity. Evaluation on Switchboard using intrinsic metrics - global gap statistics, correlations between consecutive gaps, copula-based higher-order dependencies, turn-taking entropy, and gap survival functions - shows that speaker-aware simulation better aligns with real conversational patterns than the baseline method, capturing fine-grained temporal dependencies and realistic speaker alternation, while revealing open challenges in modeling long-range conversational structure.

Method: The authors construct CompSpoof dataset covering various combinations of bona fide and spoofed speech/environmental sounds. They propose a separation-enhanced joint learning framework that separates audio components and applies anti-spoofing models to each component individually while using joint learning to preserve detection-relevant information.

Result: Extensive experiments show that the proposed method outperforms baseline approaches, demonstrating the necessity of separating audio components and detecting spoofing for each component separately.

Conclusion: Component-level spoofing detection requires specialized approaches that can handle partial manipulations within audio signals. The proposed framework effectively addresses this challenge and highlights the importance of component-wise analysis for accurate spoofing detection.

Abstract: Component-level audio Spoofing (Comp-Spoof) targets a new form of audio manipulation where only specific components of a signal, such as speech or environmental sound, are forged or substituted while other components remain genuine. Existing anti-spoofing datasets and methods treat an utterance or a segment as entirely bona fide or entirely spoofed, and thus cannot accurately detect component-level spoofing. To address this, we construct a new dataset, CompSpoof, covering multiple combinations of bona fide and spoofed speech and environmental sound. We further propose a separation-enhanced joint learning framework that separates audio components apart and applies anti-spoofing models to each one. Joint learning is employed, preserving information relevant for detection. Extensive experiments demonstrate that our method outperforms the baseline, highlighting the necessity of separate components and the importance of detecting spoofing for each component separately. Datasets and code are available at: https://github.com/XuepingZhang/CompSpoof.

Method: DART implements acoustic radiance transfer (ART) by discretizing the time-dependent rendering equation, modeling time- and direction-dependent energy exchange between surface patches with flexible material properties.

Result: DART shows better generalization under sparse measurement scenarios compared to signal processing and neural network baselines, while remaining a simple and fully interpretable system.

Conclusion: DART provides an effective differentiable framework for acoustic field learning that combines the benefits of interpretability with improved performance in sparse data settings.

Abstract: Geometric acoustics is an efficient approach to room acoustics modeling, governed by the canonical time-dependent rendering equation. Acoustic radiance transfer (ART) solves the equation through discretization, modeling the time- and direction-dependent energy exchange between surface patches given with flexible material properties. We introduce DART, a differentiable and efficient implementation of ART that enables gradient-based optimization of material properties. We evaluate DART on a simpler variant of the acoustic field learning task, which aims to predict the energy responses of novel source-receiver settings. Experimental results show that DART exhibits favorable properties, e.g., better generalization under a sparse measurement scenario, compared to existing signal processing and neural network baselines, while remaining a simple, fully interpretable system.

Method: Proposes DISPatch framework using Knowledge Gap Score to identify patches where teacher outperforms student, and Multi-Scale Selective Patches (MSSP) using different patch sizes across frequency bands to handle spectral heterogeneity.

Result: DISPatch integrated into conventional KD methods shows consistent gains in compact students. Combined with MSSP in state-of-the-art frequency-dependent KD method significantly improves performance across all metrics.

Conclusion: Selective distillation focusing on knowledge gaps rather than full imitation leads to more effective knowledge transfer and better performance in speech enhancement tasks.

Abstract: In speech enhancement, knowledge distillation (KD) compresses models by transferring a high-capacity teacher’s knowledge to a compact student. However, conventional KD methods train the student to mimic the teacher’s output entirely, which forces the student to imitate the regions where the teacher performs poorly and to apply distillation to the regions where the student already performs well, which yields only marginal gains. We propose Distilling Selective Patches (DISPatch), a KD framework for speech enhancement that applies the distillation loss to spectrogram patches where the teacher outperforms the student, as determined by a Knowledge Gap Score. This approach guides optimization toward areas with the most significant potential for student improvement while minimizing the influence of regions where the teacher may provide unreliable instruction. Furthermore, we introduce Multi-Scale Selective Patches (MSSP), a frequency-dependent method that uses different patch sizes across low- and high-frequency bands to account for spectral heterogeneity. We incorporate DISPatch into conventional KD methods and observe consistent gains in compact students. Moreover, integrating DISPatch and MSSP into a state-of-the-art frequency-dependent KD method considerably improves performance across all metrics.

Method: Construct a full mixing console with all processors applied to every track, use differentiable implementations for gradient-based optimization, iteratively remove negligible processors while fine-tuning remaining ones, and implement parallel batch processing with dry/wet parameter exploitation for efficiency.

Result: The method preserves full mixing console quality while removing approximately two-thirds of processors, and enables efficient large-scale graph data collection for downstream tasks.

Conclusion: The proposed approach effectively reverse engineers music mixes through processor graph optimization, with practical applications in mix analysis and automatic mixing systems, supported by extensive quantitative and qualitative validation.

Abstract: Reverse engineering of music mixes aims to uncover how dry source signals are processed and combined to produce a final mix. We extend the prior works to reflect the compositional nature of mixing and search for a graph of audio processors. First, we construct a mixing console, applying all available processors to every track and subgroup. With differentiable processor implementations, we optimize their parameters with gradient descent. Then, we repeat the process of removing negligible processors and fine-tuning the remaining ones. This way, the quality of the full mixing console can be preserved while removing approximately two-thirds of the processors. The proposed method can be used not only to analyze individual music mixes but also to collect large-scale graph data that can be used for downstream tasks, e.g., automatic mixing. Especially for the latter purpose, efficient implementation of the search is crucial. To this end, we present an efficient batch-processing method that computes multiple processors in parallel. We also exploit the “dry/wet” parameter of the processors to accelerate the search. Extensive quantitative and qualitative analyses are conducted to evaluate the proposed method’s performance, behavior, and computational cost.

Method: COSE introduces a velocity composition identity to compute average velocity efficiently, eliminating the need for expensive Jacobian-vector product computations while preserving theoretical consistency. This enables one-step flow matching for speech enhancement.

Result: Extensive experiments show COSE delivers up to 5x faster sampling and reduces training cost by 40% compared to existing methods, without compromising speech quality on standard benchmarks.

Conclusion: COSE provides an efficient one-step flow matching framework for speech enhancement that significantly reduces computational costs while maintaining competitive enhancement quality, making it a practical alternative to multi-step generative models.

Abstract: Diffusion and flow matching (FM) models have achieved remarkable progress in speech enhancement (SE), yet their dependence on multi-step generation is computationally expensive and vulnerable to discretization errors. Recent advances in one-step generative modeling, particularly MeanFlow, provide a promising alternative by reformulating dynamics through average velocity fields. In this work, we present COSE, a one-step FM framework tailored for SE. To address the high training overhead of Jacobian-vector product (JVP) computations in MeanFlow, we introduce a velocity composition identity to compute average velocity efficiently, eliminating expensive computation while preserving theoretical consistency and achieving competitive enhancement quality. Extensive experiments on standard benchmarks show that COSE delivers up to 5x faster sampling and reduces training cost by 40%, all without compromising speech quality. Code is available at https://github.com/ICDM-UESTC/COSE.

Method: Proposes Fed-PISA with disentangled Low-Rank Adaptation: private ID-LoRA for local timbre retention and lightweight style-LoRA for server transmission. Uses collaborative filtering-inspired aggregation to create custom models by learning from stylistically similar peers.

Result: Experiments show Fed-PISA improves style expressivity, naturalness, and speaker similarity while outperforming standard federated baselines with minimal communication costs.

Conclusion: Fed-PISA effectively addresses communication cost and heterogeneity issues in federated voice cloning, achieving better personalization and performance.

Abstract: Voice cloning for Text-to-Speech (TTS) aims to generate expressive and personalized speech from text using limited data from a target speaker. Federated Learning (FL) offers a collaborative and privacy-preserving framework for this task, but existing approaches suffer from high communication costs and tend to suppress stylistic heterogeneity, resulting in insufficient personalization. To address these issues, we propose Fed-PISA, which stands for Federated Personalized Identity-Style Adaptation. To minimize communication costs, Fed-PISA introduces a disentangled Low-Rank Adaptation (LoRA) mechanism: the speaker’s timbre is retained locally through a private ID-LoRA, while only a lightweight style-LoRA is transmitted to the server, thereby minimizing parameter exchange. To harness heterogeneity, our aggregation method, inspired by collaborative filtering, is introduced to create custom models for each client by learning from stylistically similar peers. Experiments show that Fed-PISA improves style expressivity, naturalness, and speaker similarity, outperforming standard federated baselines with minimal communication costs.

Method: Combines multi-stage causal distillation of WavLM with focal modulation architecture, including a lightweight refiner module to enhance quality under latency constraints. Compresses speech into a single binary codebook.

Result: Outperforms existing streamable codecs at comparable bitrates (0.55-0.80 kbps) with 80ms theoretical latency, while preserving both semantic and acoustic information.

Conclusion: FocalCodec-Stream achieves a favorable trade-off between reconstruction quality, downstream task performance, latency, and efficiency, making it suitable for real-time applications.

Abstract: Neural audio codecs are a fundamental component of modern generative audio pipelines. Although recent codecs achieve strong low-bitrate reconstruction and provide powerful representations for downstream tasks, most are non-streamable, limiting their use in real-time applications. We present FocalCodec-Stream, a hybrid codec based on focal modulation that compresses speech into a single binary codebook at 0.55 - 0.80 kbps with a theoretical latency of 80 ms. Our approach combines multi-stage causal distillation of WavLM with targeted architectural improvements, including a lightweight refiner module that enhances quality under latency constraints. Experiments show that FocalCodec-Stream outperforms existing streamable codecs at comparable bitrates, while preserving both semantic and acoustic information. The result is a favorable trade-off between reconstruction quality, downstream task performance, latency, and efficiency. Code and checkpoints will be released at https://github.com/lucadellalib/focalcodec.

Method: Combines energy-based peak detection with convolutional neural networks trained on a dataset of 3,396 bounce samples across 10 racket configurations for real-time processing.

Result: Achieves accurate, low-latency bounce detection and reliable classification of contact surface and spin application from audio alone.

Conclusion: The audio-based approach enables new possibilities for spin estimation in robotics and real-time coaching feedback, with publicly released dataset and code.

Abstract: Sound can complement vision in ball sports by providing subtle cues about contact dynamics. In table tennis, the brief, high-frequency sounds produced during racket-ball impacts carry information about the racket type, the surface contacted, and whether spin was applied. We address three key problems in this domain: (1) precise bounce detection with millisecond-level temporal accuracy, (2) classification of bounce surface (e.g., racket, table, floor), and (3) spin detection from audio alone. To this end, we propose a real-time-capable pipeline that combines energy-based peak detection with convolutional neural networks trained on a novel dataset of 3,396 bounce samples recorded across 10 racket configurations. The system achieves accurate and low-latency detection of bounces, and reliably classifies both the surface of contact and whether spin was applied. This audio-based approach opens up new possibilities for spin estimation in robotic systems and for real-time feedback in coaching tools. We publicly release both the dataset and code to support further research.

Method: Used the QDAT dataset with over 1,500 audio recordings converted to normalized mel-spectrograms. Applied EfficientNet-B0 architecture enhanced with Squeeze-and-Excitation attention mechanism for classification of three Tajweed rules.

Result: The model achieved accuracy rates of 95.35% for Al Mad, 99.34% for Ghunnah, and 97.01% for Ikhfaa. Learning curve analysis confirmed model robustness without overfitting.

Conclusion: The proposed approach demonstrates high efficiency and effectiveness, paving the way for developing interactive educational systems for automated Tajweed study and evaluation.

Abstract: Proper recitation of the Quran, adhering to the rules of Tajweed, is crucial for preventing mistakes during recitation and requires significant effort to master. Traditional methods of teaching these rules are limited by the availability of qualified instructors and time constraints. Automatic evaluation of recitation can address these challenges by providing prompt feedback and supporting independent practice. This study focuses on developing a deep learning model to classify three Tajweed rules - separate stretching (Al Mad), tight noon (Ghunnah), and hide (Ikhfaa) - using the publicly available QDAT dataset, which contains over 1,500 audio recordings. The input data consisted of audio recordings from this dataset, transformed into normalized mel-spectrograms. For classification, the EfficientNet-B0 architecture was used, enhanced with a Squeeze-and-Excitation attention mechanism. The developed model achieved accuracy rates of 95.35%, 99.34%, and 97.01% for the respective rules. An analysis of the learning curves confirmed the model’s robustness and absence of overfitting. The proposed approach demonstrates high efficiency and paves the way for developing interactive educational systems for Tajweed study.

Method: Two variations: 1) descriptive model conditioned on natural language prompts, 2) parametric model conditioned on both text and spatial trajectory parameters. Trained on a new dataset of over 1 million simulated FOA caption pairs with annotated azimuth, elevation, and motion attributes.

Result: Achieves state-of-the-art semantic alignment and perceptual quality comparable to leading text-to-audio systems, while uniquely attaining low spatial localization error.

Conclusion: SonicMotion represents a significant advancement in spatial audio generation by enabling explicit control over moving sound sources in FOA format, addressing limitations of existing static-only models.

Abstract: Spatial audio has become central to immersive applications such as VR/AR, cinema, and music. Existing generative audio models are largely limited to mono or stereo formats and cannot capture the full 3D localization cues available in first-order Ambisonics (FOA). Recent FOA models extend text-to-audio generation but remain restricted to static sources. In this work, we introduce SonicMotion, the first end-to-end latent diffusion framework capable of generating FOA audio with explicit control over moving sound sources. SonicMotion is implemented in two variations: 1) a descriptive model conditioned on natural language prompts, and 2) a parametric model conditioned on both text and spatial trajectory parameters for higher precision. To support training and evaluation, we construct a new dataset of over one million simulated FOA caption pairs that include both static and dynamic sources with annotated azimuth, elevation, and motion attributes. Experiments show that SonicMotion achieves state-of-the-art semantic alignment and perceptual quality comparable to leading text-to-audio systems, while uniquely attaining low spatial localization error.

Method: The method involves using agglomerative hierarchical clustering to assign pseudo-attribute labels based on representations from a domain-adaptive pre-trained model, followed by supervised fine-tuning of the pre-trained model for machine attribute classification.

Result: The proposed approach achieves new state-of-the-art performance on the DCASE 2025 Challenge dataset, demonstrating significant performance gains and outperforming the authors’ previous top-ranking system in the challenge.

Conclusion: The agglomerative hierarchical clustering method for pseudo-attribute label assignment combined with model adaptation through supervised fine-tuning provides an effective solution for ASD when machine attribute labels are missing, yielding superior performance compared to existing approaches.

Abstract: Anomalous Sound Detection (ASD) is often formulated as a machine attribute classification task, a strategy necessitated by the common scenario where only normal data is available for training. However, the exhaustive collection of machine attribute labels is laborious and impractical. To address the challenge of missing attribute labels, this paper proposes an agglomerative hierarchical clustering method for the assignment of pseudo-attribute labels using representations derived from a domain-adaptive pre-trained model, which are expected to capture machine attribute characteristics. We then apply model adaptation to this pre-trained model through supervised fine-tuning for machine attribute classification, resulting in a new state-of-the-art performance. Evaluation on the Detection and Classification of Acoustic Scenes and Events (DCASE) 2025 Challenge dataset demonstrates that our proposed approach yields significant performance gains, ultimately outperforming our previous top-ranking system in the challenge.

Method: Evaluated multiple CNN architectures including PANNs 1D/2D models, ConvNeXt-based audio classification model, MobileNetV3, and PANNs-derived CNN9/CNN13. All models converted to ONNX format for portability. Conducted continuous 24-hour inference sessions to assess performance stability.

Result: Appropriate model selection and optimization can maintain consistent inference latency and effectively manage thermal behavior over extended periods on Raspberry Pi.

Conclusion: The findings provide valuable insights for deploying audio tagging models in real-world edge computing scenarios, demonstrating feasibility of stable long-term operation on constrained hardware.

Abstract: Convolutional Neural Networks (CNNs) have demonstrated exceptional performance in audio tagging tasks. However, deploying these models on resource-constrained devices like the Raspberry Pi poses challenges related to computational efficiency and thermal management. In this paper, a comprehensive evaluation of multiple convolutional neural network (CNN) architectures for audio tagging on the Raspberry Pi is conducted, encompassing all 1D and 2D models from the Pretrained Audio Neural Networks (PANNs) framework, a ConvNeXt-based model adapted for audio classification, as well as MobileNetV3 architectures. In addition, two PANNs-derived networks, CNN9 and CNN13, recently proposed, are also evaluated. To enhance deployment efficiency and portability across diverse hardware platforms, all models are converted to the Open Neural Network Exchange (ONNX) format. Unlike previous works that focus on a single model, our analysis encompasses a broader range of architectures and involves continuous 24-hour inference sessions to assess performance stability. Our experiments reveal that, with appropriate model selection and optimization, it is possible to maintain consistent inference latency and manage thermal behavior effectively over extended periods. These findings provide valuable insights for deploying audio tagging models in real-world edge computing scenarios.

Method: The model learns the average velocity over finite intervals using a Jacobian-vector product to instantiate the MeanFlow identity, enabling direct supervision of finite-interval displacement while maintaining consistency with instantaneous-field constraints.

Result: On VoiceBank-DEMAND, the single-step model achieves strong intelligibility, fidelity, and perceptual quality with substantially lower computational cost than multistep baselines.

Conclusion: MeanFlowSE provides an efficient, high-fidelity framework for real-time generative speech enhancement without requiring knowledge distillation or external teachers.

Abstract: Multistep inference is a bottleneck for real-time generative speech enhancement because flow- and diffusion-based systems learn an instantaneous velocity field and therefore rely on iterative ordinary differential equation (ODE) solvers. We introduce MeanFlowSE, a conditional generative model that learns the average velocity over finite intervals along a trajectory. Using a Jacobian-vector product (JVP) to instantiate the MeanFlow identity, we derive a local training objective that directly supervises finite-interval displacement while remaining consistent with the instantaneous-field constraint on the diagonal. At inference, MeanFlowSE performs single-step generation via a backward-in-time displacement, removing the need for multistep solvers; an optional few-step variant offers additional refinement. On VoiceBank-DEMAND, the single-step model achieves strong intelligibility, fidelity, and perceptual quality with substantially lower computational cost than multistep baselines. The method requires no knowledge distillation or external teachers, providing an efficient, high-fidelity framework for real-time generative speech enhancement. The proposed method is open-sourced at https://github.com/liduojia1/MeanFlowSE.

Method: Systematic evaluation by freezing/selectively updating visual encoder, scaling decoder size, comparing adaptation strategies and architectures, and varying training data across LRS2, LRS3, and their combination.

Result: Llama-2-13B model trained on combined dataset achieves 24.7% WER on LRS3 and 47.0% on WildVSR (SOTA without additional supervision), with gains primarily from lexical rather than semantic processing.

Conclusion: LLM decoders refine contextual reasoning rather than visual features, emphasizing the need for stronger visual encoders to drive meaningful progress in VSR.

Abstract: Advances in self-supervised encoders have improved Visual Speech Recognition (VSR). Recent approaches integrating these encoders with LLM decoders improves transcription accuracy; however, it remains unclear whether these gains stem from visual understanding or stronger language modeling. In this work, we systematically evaluate LLM decoders by freezing or selectively updating the visual encoder, scaling decoder size, comparing adaptation strategies and architectures, and varying training data across LRS2, LRS3, and their combination. Evaluation on LRS2, LRS3, and WildVSR shows that scaling and adaptation yield limited improvements, while combining datasets enhances generalization. Semantic analysis reveals that gains arise primarily from lexical rather than semantic processing. Our Llama-2-13B model trained on the combined set achieves 24.7% WER on LRS3 and 47.0% on WildVSR, establishing SOTA among models trained without additional supervision. Our findings indicate LLM decoders refine contextual reasoning rather than visual features, emphasizing the need for stronger visual encoders to drive meaningful progress.

Method: Unified prompt-based framework that binds class-level semantics to dedicated prompt tokens during training, allowing instant unlearning through token removal without accessing original data.

Result: Preserves predictive performance on retained classes while effectively erasing forgotten ones, resistant to membership inference attacks and prevents residual knowledge extraction.

Conclusion: Embeds removability into architecture design, establishing modular, scalable and ethically responsive AI models suitable for sensitive and regulated environments.

Abstract: Foundation models have transformed multimedia analysis by enabling robust and transferable representations across diverse modalities and tasks. However, their static deployment conflicts with growing societal and regulatory demands – particularly the need to unlearn specific data upon request, as mandated by privacy frameworks such as the GDPR. Traditional unlearning approaches, including retraining, activation editing, or distillation, are often computationally expensive, fragile, and ill-suited for real-time or continuously evolving systems. In this paper, we propose a paradigm shift: rethinking unlearning not as a retroactive intervention but as a built-in capability. We introduce a prompt-based learning framework that unifies knowledge acquisition and removal within a single training phase. Rather than encoding information in model weights, our approach binds class-level semantics to dedicated prompt tokens. This design enables instant unlearning simply by removing the corresponding prompt – without retraining, model modification, or access to original data. Experiments demonstrate that our framework preserves predictive performance on retained classes while effectively erasing forgotten ones. Beyond utility, our method exhibits strong privacy and security guarantees: it is resistant to membership inference attacks, and prompt removal prevents any residual knowledge extraction, even under adversarial conditions. This ensures compliance with data protection principles and safeguards against unauthorized access to forgotten information, making the framework suitable for deployment in sensitive and regulated environments. Overall, by embedding removability into the architecture itself, this work establishes a new foundation for designing modular, scalable and ethically responsive AI models.

Method: Proposes MPNP-DDI with a unique message-passing scheme that learns hierarchical graph representations at multiple scales, a cross-drug co-attention mechanism for dynamic fusion of multi-scale representations, and an integrated neural process module for uncertainty estimation.

Result: Extensive experiments show MPNP-DDI significantly outperforms state-of-the-art baselines on benchmark datasets.

Conclusion: MPNP-DDI provides accurate, generalizable, and uncertainty-aware predictions, representing a powerful computational tool for pharmacovigilance, polypharmacy risk assessment, and precision medicine.

Abstract: Accurate prediction of drug-drug interactions (DDI) is crucial for medication safety and effective drug development. However, existing methods often struggle to capture structural information across different scales, from local functional groups to global molecular topology, and typically lack mechanisms to quantify prediction confidence. To address these limitations, we propose MPNP-DDI, a novel Multi-scale Graph Neural Process framework. The core of MPNP-DDI is a unique message-passing scheme that, by being iteratively applied, learns a hierarchy of graph representations at multiple scales. Crucially, a cross-drug co-attention mechanism then dynamically fuses these multi-scale representations to generate context-aware embeddings for interacting drug pairs, while an integrated neural process module provides principled uncertainty estimation. Extensive experiments demonstrate that MPNP-DDI significantly outperforms state-of-the-art baselines on benchmark datasets. By providing accurate, generalizable, and uncertainty-aware predictions built upon multi-scale structural features, MPNP-DDI represents a powerful computational tool for pharmacovigilance, polypharmacy risk assessment, and precision medicine.

Method: The survey investigates GenAI-wireless sensing convergence from two perspectives: 1) Integration modes (plugin for task-specific models vs. direct solver for sensing tasks), and 2) Analysis of mainstream generative models (GANs, VAEs, diffusion models) and their applicability to various wireless sensing tasks.

Result: The study identifies how generative techniques like data augmentation, domain adaptation, and denoising can significantly improve wireless sensing applications. It analyzes the unique advantages of different generative models across various sensing tasks.

Conclusion: Key challenges in applying GenAI to wireless sensing are identified, with a future direction proposed toward developing a wireless foundation model - a unified, pre-trained design capable of scalable, adaptable, and efficient signal understanding across diverse sensing tasks.

Abstract: Generative Artificial Intelligence (GenAI) has made significant advancements in fields such as computer vision (CV) and natural language processing (NLP), demonstrating its capability to synthesize high-fidelity data and improve generalization. Recently, there has been growing interest in integrating GenAI into wireless sensing systems. By leveraging generative techniques such as data augmentation, domain adaptation, and denoising, wireless sensing applications, including device localization, human activity recognition, and environmental monitoring, can be significantly improved. This survey investigates the convergence of GenAI and wireless sensing from two complementary perspectives. First, we explore how GenAI can be integrated into wireless sensing pipelines, focusing on two modes of integration: as a plugin to augment task-specific models and as a solver to directly address sensing tasks. Second, we analyze the characteristics of mainstream generative models, such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and diffusion models, and discuss their applicability and unique advantages across various wireless sensing tasks. We further identify key challenges in applying GenAI to wireless sensing and outline a future direction toward a wireless foundation model: a unified, pre-trained design capable of scalable, adaptable, and efficient signal understanding across diverse sensing tasks.

Method: Proposes IEFS-GMB which constructs a dynamic memory bank storing historical gradients, computes feature importance via information entropy, and applies entropy-based weighting to select informative EEG features.

Result: Experiments on four public neurological disease datasets show accuracy improvements of 0.64% to 6.45% over baseline models, outperforming four competing FS techniques while improving interpretability.

Conclusion: IEFS-GMB effectively enhances EEG classification performance and interpretability, supporting practical clinical applications for automated neurological disorder detection.

Abstract: Deep learning-based EEG classification is crucial for the automated detection of neurological disorders, improving diagnostic accuracy and enabling early intervention. However, the low signal-to-noise ratio of EEG signals limits model performance, making feature selection (FS) vital for optimizing representations learned by neural network encoders. Existing FS methods are seldom designed specifically for EEG diagnosis; many are architecture-dependent and lack interpretability, limiting their applicability. Moreover, most rely on single-iteration data, resulting in limited robustness to variability. To address these issues, we propose IEFS-GMB, an Information Entropy-based Feature Selection method guided by a Gradient Memory Bank. This approach constructs a dynamic memory bank storing historical gradients, computes feature importance via information entropy, and applies entropy-based weighting to select informative EEG features. Experiments on four public neurological disease datasets show that encoders enhanced with IEFS-GMB achieve accuracy improvements of 0.64% to 6.45% over baseline models. The method also outperforms four competing FS techniques and improves model interpretability, supporting its practical use in clinical settings.

Method: Combined slang term curation with biomedical concept extraction via MetaMap to identify 92,000+ tweets, used expert-guided heuristic labeling for polarity, and trained machine learning classifiers with cost-sensitive learning and oversampling techniques.

Result: Achieved top performance with eXtreme Gradient Boosting (F1 = 0.885, AUPRC = 0.934), enabling detection of substance-specific phenotypic effects from social media data.

Conclusion: Twitter data and polarity classification models can effectively support real-time pharmacovigilance and drug effect characterization with high accuracy.

Abstract: Understanding the real-world effects of recreational drug use remains a critical challenge in public health and biomedical research, especially as traditional surveillance systems often underrepresent user experiences. In this study, we leverage social media (specifically Twitter) as a rich and unfiltered source of user-reported effects associated with three emerging psychoactive substances: ecstasy, GHB, and 2C-B. By combining a curated list of slang terms with biomedical concept extraction via MetaMap, we identified and weakly annotated over 92,000 tweets mentioning these substances. Each tweet was labeled with a polarity reflecting whether it reported a positive or negative effect, following an expert-guided heuristic process. We then performed descriptive and comparative analyses of the reported phenotypic outcomes across substances and trained multiple machine learning classifiers to predict polarity from tweet content, accounting for strong class imbalance using techniques such as cost-sensitive learning and synthetic oversampling. The top performance on the test set was obtained from eXtreme Gradient Boosting with cost-sensitive learning (F1 = 0.885, AUPRC = 0.934). Our findings reveal that Twitter enables the detection of substance-specific phenotypic effects, and that polarity classification models can support real-time pharmacovigilance and drug effect characterization with high accuracy.

Method: Represent Transformer LLMs as directed weighted graphs (nodes=components, edges=causal influence), track evolution across 143 training checkpoints using graph-theoretic metrics.

Result: Network structure evolves through exploration, consolidation, and refinement phases, with emergence of stable information spreaders and dynamic information gatherers.

Conclusion: Component-level network perspective provides macroscopic understanding of self-organizing principles driving functional circuit formation in LLMs.

Abstract: The process by which Large Language Models (LLMs) acquire complex capabilities during training remains a key open question in mechanistic interpretability. This project investigates whether these learning dynamics can be characterized through the lens of Complex Network Theory (CNT). I introduce a novel methodology to represent a Transformer-based LLM as a directed, weighted graph where nodes are the model’s computational components (attention heads and MLPs) and edges represent causal influence, measured via an intervention-based ablation technique. By tracking the evolution of this component-graph across 143 training checkpoints of the Pythia-14M model on a canonical induction task, I analyze a suite of graph-theoretic metrics. The results reveal that the network’s structure evolves through distinct phases of exploration, consolidation, and refinement. Specifically, I identify the emergence of a stable hierarchy of information spreader components and a dynamic set of information gatherer components, whose roles reconfigure at key learning junctures. This work demonstrates that a component-level network perspective offers a powerful macroscopic lens for visualizing and understanding the self-organizing principles that drive the formation of functional circuits in LLMs.

Method: Developed a machine learning model using graph neural networks to predict which pricing problems are likely to produce columns with negative reduced cost, enabling a more efficient partial column generation approach.

Result: Computational experiments show the proposed strategy enhances solution methods and outperforms traditional partial column generation approaches, particularly on hard instances solved under tight time limits.

Conclusion: The machine learning-based partial column generation strategy is effective for improving the efficiency of solving team formation and routing problems, especially in time-constrained scenarios.

Abstract: The team formation and routing problem is a challenging optimization problem with several real-world applications in fields such as airport, healthcare, and maintenance operations. To solve this problem, exact solution methods based on column generation have been proposed in the literature. In this paper, we propose a novel partial column generation strategy for settings with multiple pricing problems, based on predicting which ones are likely to yield columns with a negative reduced cost. We develop a machine learning model tailored to the team formation and routing problem that leverages graph neural networks for these predictions. Computational experiments demonstrate that applying our strategy enhances the solution method and outperforms traditional partial column generation approaches from the literature, particularly on hard instances solved under a tight time limit.

Method: Three complementary innovations: 1) Reasoning-Oriented Data Strategy combining curated medical QA datasets with knowledge-graph-guided synthesis, 2) Chain-of-Thought cold start to distill reasoning trajectories, 3) Two-stage Reinforcement Learning from Verifiable Rewards using Group Relative Policy Optimization.

Result: Fleming-R1 delivers substantial parameter-efficient improvements: 7B variant surpasses larger baselines, 32B model achieves near-parity with GPT-4o and consistently outperforms strong open-source alternatives across diverse medical benchmarks.

Conclusion: Structured data design, reasoning-oriented initialization, and verifiable reinforcement learning can advance clinical reasoning beyond simple accuracy optimization, enabling safer deployment in high-stakes clinical environments.

Abstract: While large language models show promise in medical applications, achieving expert-level clinical reasoning remains challenging due to the need for both accurate answers and transparent reasoning processes. To address this challenge, we introduce Fleming-R1, a model designed for verifiable medical reasoning through three complementary innovations. First, our Reasoning-Oriented Data Strategy (RODS) combines curated medical QA datasets with knowledge-graph-guided synthesis to improve coverage of underrepresented diseases, drugs, and multi-hop reasoning chains. Second, we employ Chain-of-Thought (CoT) cold start to distill high-quality reasoning trajectories from teacher models, establishing robust inference priors. Third, we implement a two-stage Reinforcement Learning from Verifiable Rewards (RLVR) framework using Group Relative Policy Optimization, which consolidates core reasoning skills while targeting persistent failure modes through adaptive hard-sample mining. Across diverse medical benchmarks, Fleming-R1 delivers substantial parameter-efficient improvements: the 7B variant surpasses much larger baselines, while the 32B model achieves near-parity with GPT-4o and consistently outperforms strong open-source alternatives. These results demonstrate that structured data design, reasoning-oriented initialization, and verifiable reinforcement learning can advance clinical reasoning beyond simple accuracy optimization. We release Fleming-R1 publicly to promote transparent, reproducible, and auditable progress in medical AI, enabling safer deployment in high-stakes clinical environments.

Method: Proposes a hybrid unary-binary architecture that eliminates costly encoders and enables multiplier-less MLP execution. Introduces architecture-aware training to optimize area and power efficiency.

Result: Evaluation on six datasets shows average reductions of 46% in area and 39% in power, with minimal accuracy loss, outperforming other state-of-the-art MLP designs.

Conclusion: The hybrid architecture successfully addresses the limitations of printed electronics for ML applications, achieving substantial efficiency improvements while maintaining accuracy.

Abstract: Printed Electronics (PE) provide a flexible, cost-efficient alternative to silicon for implementing machine learning (ML) circuits, but their large feature sizes limit classifier complexity. Leveraging PE’s low fabrication and NRE costs, designers can tailor hardware to specific ML models, simplifying circuit design. This work explores alternative arithmetic and proposes a hybrid unary-binary architecture that removes costly encoders and enables efficient, multiplier-less execution of MLP classifiers. We also introduce architecture-aware training to further improve area and power efficiency. Evaluation on six datasets shows average reductions of 46% in area and 39% in power, with minimal accuracy loss, surpassing other state-of-the-art MLP designs.

Method: Forward process uses synchronization among phase variables via coupled oscillator interactions, collapsing data into low-entropy von Mises distribution. Reverse process performs desynchronization with learned score function, using wrapped Gaussian kernels and periodicity-aware networks.

Result: Competitive results on general image benchmarks and significant improvement on orientation-dense datasets like fingerprints and textures.

Conclusion: Biologically inspired synchronization dynamics show promise as structured priors in generative modeling, enabling hierarchical generation from global coherence to fine-scale details.

Abstract: Orientation-rich images, such as fingerprints and textures, often exhibit coherent angular directional patterns that are challenging to model using standard generative approaches based on isotropic Euclidean diffusion. Motivated by the role of phase synchronization in biological systems, we propose a score-based generative model built on periodic domains by leveraging stochastic Kuramoto dynamics in the diffusion process. In neural and physical systems, Kuramoto models capture synchronization phenomena across coupled oscillators – a behavior that we re-purpose here as an inductive bias for structured image generation. In our framework, the forward process performs \textit{synchronization} among phase variables through globally or locally coupled oscillator interactions and attraction to a global reference phase, gradually collapsing the data into a low-entropy von Mises distribution. The reverse process then performs \textit{desynchronization}, generating diverse patterns by reversing the dynamics with a learned score function. This approach enables structured destruction during forward diffusion and a hierarchical generation process that progressively refines global coherence into fine-scale details. We implement wrapped Gaussian transition kernels and periodicity-aware networks to account for the circular geometry. Our method achieves competitive results on general image benchmarks and significantly improves generation quality on orientation-dense datasets like fingerprints and textures. Ultimately, this work demonstrates the promise of biologically inspired synchronization dynamics as structured priors in generative modeling.

Method: Proposes Global Pre-fixing, Local Adjusting strategy that uses ETF to create non-overlapping regions for different tasks globally, while allowing intra-task feature adjustment within allocated regions.

Result: Extensive experiments validate the effectiveness of the method in ensuring discriminative feature structures both between and within tasks.

Conclusion: GPLASC can be seamlessly integrated into existing contrastive continual learning frameworks and simultaneously addresses inter-task and intra-task feature confusion.

Abstract: Continual learning (CL) involves acquiring and accumulating knowledge from evolving tasks while alleviating catastrophic forgetting. Recently, leveraging contrastive loss to construct more transferable and less forgetful representations has been a promising direction in CL. Despite advancements, their performance is still limited due to confusion arising from both inter-task and intra-task features. To address the problem, we propose a simple yet effective contrastive strategy named \textbf{G}lobal \textbf{P}re-fixing, \textbf{L}ocal \textbf{A}djusting for \textbf{S}upervised \textbf{C}ontrastive learning (GPLASC). Specifically, to avoid task-level confusion, we divide the entire unit hypersphere of representations into non-overlapping regions, with the centers of the regions forming an inter-task pre-fixed \textbf{E}quiangular \textbf{T}ight \textbf{F}rame (ETF). Meanwhile, for individual tasks, our method helps regulate the feature structure and form intra-task adjustable ETFs within their respective allocated regions. As a result, our method \textit{simultaneously} ensures discriminative feature structures both between tasks and within tasks and can be seamlessly integrated into any existing contrastive continual learning framework. Extensive experiments validate its effectiveness.

Method: Introduces Small Sample Beta Correction (SSBC), which adjusts the conformal significance level using the exact finite-sample distribution of conformal coverage to provide probabilistic guarantees.

Result: SSBC ensures that with user-defined probability over the calibration draw, the deployed predictor achieves at least the desired coverage level.

Conclusion: SSBC addresses the variance problem in split conformal prediction by providing probabilistic coverage guarantees rather than just expected coverage, making conformal prediction more reliable for practical risk control applications.

Abstract: Conformal prediction provides distribution-free prediction sets with guaranteed marginal coverage. However, in split conformal prediction this guarantee is training-conditional only in expectation: across many calibration draws, the average coverage equals the nominal level, but the realized coverage for a single calibration set may vary substantially. This variance undermines effective risk control in practical applications. Here we introduce the Small Sample Beta Correction (SSBC), a plug-and-play adjustment to the conformal significance level that leverages the exact finite-sample distribution of conformal coverage to provide probabilistic guarantees, ensuring that with user-defined probability over the calibration draw, the deployed predictor achieves at least the desired coverage.

Method: Conducted large-scale empirical study of LLMs’ zero-shot capabilities across diverse tabular prediction tasks, analyzing performance variability and developing metrics to predict task-level suitability without labeled data.

Result: LLM performance is highly variable within and across datasets, but predicted probabilities become strong accuracy signals when base performance is good. Some metrics effectively predict LLM suitability for new tasks without requiring labeled data.

Conclusion: LLMs can be effectively deployed for zero-shot prediction when task suitability is properly assessed using proposed metrics, enabling users to identify tasks where LLMs will perform well versus those where they are unsuitable.

Abstract: Recent work has investigated the capabilities of large language models (LLMs) as zero-shot models for generating individual-level characteristics (e.g., to serve as risk models or augment survey datasets). However, when should a user have confidence that an LLM will provide high-quality predictions for their particular task? To address this question, we conduct a large-scale empirical study of LLMs’ zero-shot predictive capabilities across a wide range of tabular prediction tasks. We find that LLMs’ performance is highly variable, both on tasks within the same dataset and across different datasets. However, when the LLM performs well on the base prediction task, its predicted probabilities become a stronger signal for individual-level accuracy. Then, we construct metrics to predict LLMs’ performance at the task level, aiming to distinguish between tasks where LLMs may perform well and where they are likely unsuitable. We find that some of these metrics, each of which are assessed without labeled data, yield strong signals of LLMs’ predictive performance on new tasks.

Method: Revisits the connection between generalized derivatives and moduli of local continuity, and develops a non-uniform stochastic sample approximation approach for these moduli.

Result: Provides a practical approximation method for moduli of local continuity that can be applied to assess neural network robustness and fairness.

Conclusion: The proposed non-uniform stochastic sample approximation is important for studying neural network robustness and fairness in repeated closed-loop applications.

Abstract: Modulus of local continuity is used to evaluate the robustness of neural networks and fairness of their repeated uses in closed-loop models. Here, we revisit a connection between generalized derivatives and moduli of local continuity, and present a non-uniform stochastic sample approximation for moduli of local continuity. This is of importance in studying robustness of neural networks and fairness of their repeated uses.

Method: The authors study state-of-the-art overparameterized unfolding networks, deploy a new framework to estimate their adversarial Rademacher complexity, and provide theoretical generalization error bounds for l₂-norm constrained attacks generated by the fast gradient sign method.

Result: The paper derives tight adversarial generalization error bounds with respect to attack level, presents experimental results on real-world data that consistently corroborate the theory, and shows that overparameterization can be exploited to promote adversarial robustness.

Conclusion: This work establishes the first theoretical foundation for understanding adversarial generalization in unfolding networks, demonstrating that their overparameterization can be leveraged for robustness, which provides insights for efficiently robustifying neural networks.

Abstract: Unfolding networks are interpretable networks emerging from iterative algorithms, incorporate prior knowledge of data structure, and are designed to solve inverse problems like compressed sensing, which deals with recovering data from noisy, missing observations. Compressed sensing finds applications in critical domains, from medical imaging to cryptography, where adversarial robustness is crucial to prevent catastrophic failures. However, a solid theoretical understanding of the performance of unfolding networks in the presence of adversarial attacks is still in its infancy. In this paper, we study the adversarial generalization of unfolding networks when perturbed with $l_2$-norm constrained attacks, generated by the fast gradient sign method. Particularly, we choose a family of state-of-the-art overaparameterized unfolding networks and deploy a new framework to estimate their adversarial Rademacher complexity. Given this estimate, we provide adversarial generalization error bounds for the networks under study, which are tight with respect to the attack level. To our knowledge, this is the first theoretical analysis on the adversarial generalization of unfolding networks. We further present a series of experiments on real-world data, with results corroborating our derived theory, consistently for all data. Finally, we observe that the family’s overparameterization can be exploited to promote adversarial robustness, shedding light on how to efficiently robustify neural networks.

Method: AC-SMFG is a single-loop actor-critic algorithm that alternates between (semi-)gradient updates for the leader, a representative follower, and the mean field, operating on continuously generated Markovian samples.

Result: The algorithm establishes finite-time and finite-sample convergence to a stationary point of the Stackelberg objective, outperforming existing baselines in policy quality and convergence speed in economic environments.

Conclusion: AC-SMFG is the first Stackelberg MFG algorithm with non-asymptotic convergence guarantees, relaxing the leader-follower independence assumption through a gradient alignment condition.

Abstract: We study policy optimization in Stackelberg mean field games (MFGs), a hierarchical framework for modeling the strategic interaction between a single leader and an infinitely large population of homogeneous followers. The objective can be formulated as a structured bi-level optimization problem, in which the leader needs to learn a policy maximizing its reward, anticipating the response of the followers. Existing methods for solving these (and related) problems often rely on restrictive independence assumptions between the leader’s and followers’ objectives, use samples inefficiently due to nested-loop algorithm structure, and lack finite-time convergence guarantees. To address these limitations, we propose AC-SMFG, a single-loop actor-critic algorithm that operates on continuously generated Markovian samples. The algorithm alternates between (semi-)gradient updates for the leader, a representative follower, and the mean field, and is simple to implement in practice. We establish the finite-time and finite-sample convergence of the algorithm to a stationary point of the Stackelberg objective. To our knowledge, this is the first Stackelberg MFG algorithm with non-asymptotic convergence guarantees. Our key assumption is a “gradient alignment” condition, which requires that the full policy gradient of the leader can be approximated by a partial component of it, relaxing the existing leader-follower independence assumption. Simulation results in a range of well-established economics environments demonstrate that AC-SMFG outperforms existing multi-agent and MFG learning baselines in policy quality and convergence speed.

Method: Proposes VMDNet with: (i) sample-wise VMD to prevent leakage, (ii) frequency-aware embeddings and parallel TCNs for mode independence, and (iii) bilevel Stackelberg-inspired optimization for adaptive hyperparameter selection (K and alpha).

Result: Experiments on energy datasets show VMDNet achieves state-of-the-art performance when periodicity is strong, with clear advantages in capturing structured periodic patterns while remaining robust under weak periodicity.

Conclusion: VMDNet provides an effective causality-preserving framework that overcomes limitations of traditional VMD approaches and demonstrates superior performance in periodicity-aware time series forecasting.

Abstract: In time series forecasting, capturing recurrent temporal patterns is essential; decomposition techniques make such structure explicit and thereby improve predictive performance. Variational Mode Decomposition (VMD) is a powerful signal-processing method for periodicity-aware decomposition and has seen growing adoption in recent years. However, existing studies often suffer from information leakage and rely on inappropriate hyperparameter tuning. To address these issues, we propose VMDNet, a causality-preserving framework that (i) applies sample-wise VMD to avoid leakage; (ii) represents each decomposed mode with frequency-aware embeddings and decodes it using parallel temporal convolutional networks (TCNs), ensuring mode independence and efficient learning; and (iii) introduces a bilevel, Stackelberg-inspired optimisation to adaptively select VMD’s two core hyperparameters: the number of modes (K) and the bandwidth penalty (alpha). Experiments on two energy-related datasets demonstrate that VMDNet achieves state-of-the-art results when periodicity is strong, showing clear advantages in capturing structured periodic patterns while remaining robust under weak periodicity.

Method: The authors design novel adaptive algorithms combining momentum normalization technique with adaptive parameter choices for nonconvex-strongly-concave minimax optimization and nonconvex-strongly-convex bilevel optimization.

Result: The algorithms achieve sharp convergence rates of Õ(1/√T + √σ̄/T^{1/4}) in T iterations for gradient norm, where σ̄ is the stochastic gradient noise upper bound, without requiring prior noise knowledge.

Conclusion: This work provides the first adaptive and sharp convergence guarantees for stochastic hierarchical optimization, with experimental validation on synthetic and deep learning tasks demonstrating effectiveness.

Abstract: Hierarchical optimization refers to problems with interdependent decision variables and objectives, such as minimax and bilevel formulations. While various algorithms have been proposed, existing methods and analyses lack adaptivity in stochastic optimization settings: they cannot achieve optimal convergence rates across a wide spectrum of gradient noise levels without prior knowledge of the noise magnitude. In this paper, we propose novel adaptive algorithms for two important classes of stochastic hierarchical optimization problems: nonconvex-strongly-concave minimax optimization and nonconvex-strongly-convex bilevel optimization. Our algorithms achieve sharp convergence rates of $\widetilde{O}(1/\sqrt{T} + \sqrt{\bar{\sigma}}/T^{1/4})$ in $T$ iterations for the gradient norm, where $\bar{\sigma}$ is an upper bound on the stochastic gradient noise. Notably, these rates are obtained without prior knowledge of the noise level, thereby enabling automatic adaptivity in both low and high-noise regimes. To our knowledge, this work provides the first adaptive and sharp convergence guarantees for stochastic hierarchical optimization. Our algorithm design combines the momentum normalization technique with novel adaptive parameter choices. Extensive experiments on synthetic and deep learning tasks demonstrate the effectiveness of our proposed algorithms.

Method: Proposed Data-Augmented IBC (DA-IBC) that perturbs expert actions to create counterexamples for IBC training and uses better initialization for derivative-free inference with Energy-Based Models.

Result: DA-IBC outperforms standard IBC in CARLA simulator urban driving tasks, successfully representing multimodal action distributions that BC fails to capture.

Conclusion: DA-IBC effectively addresses the multimodality limitation of standard BC through improved energy-based modeling and data augmentation techniques.

Abstract: Standard Behavior Cloning (BC) fails to learn multimodal driving decisions, where multiple valid actions exist for the same scenario. We explore Implicit Behavioral Cloning (IBC) with Energy-Based Models (EBMs) to better capture this multimodality. We propose Data-Augmented IBC (DA-IBC), which improves learning by perturbing expert actions to form the counterexamples of IBC training and using better initialization for derivative-free inference. Experiments in the CARLA simulator with Bird’s-Eye View inputs demonstrate that DA-IBC outperforms standard IBC in urban driving tasks designed to evaluate multimodal behavior learning in a test environment. The learned energy landscapes are able to represent multimodal action distributions, which BC fails to achieve.

Method: RAMPART combines adaptive sequential halving (progressively focusing on promising features) with efficient ensembling using observation and feature subsampling, specifically tailored for top-k feature ranking.

Result: Theoretical guarantees show RAMPART achieves correct top-k ranking with high probability under mild conditions, and extensive simulations demonstrate consistent outperformance over popular feature importance methods.

Conclusion: RAMPART provides an effective framework for accurate feature ranking that explicitly optimizes for ranking accuracy rather than treating it as secondary to importance scoring.

Abstract: Accurate ranking of important features is a fundamental challenge in interpretable machine learning with critical applications in scientific discovery and decision-making. Unlike feature selection and feature importance, the specific problem of ranking important features has received considerably less attention. We introduce RAMPART (Ranked Attributions with MiniPatches And Recursive Trimming), a framework that utilizes any existing feature importance measure in a novel algorithm specifically tailored for ranking the top-$k$ features. Our approach combines an adaptive sequential halving strategy that progressively focuses computational resources on promising features with an efficient ensembling technique using both observation and feature subsampling. Unlike existing methods that convert importance scores to ranks as post-processing, our framework explicitly optimizes for ranking accuracy. We provide theoretical guarantees showing that RAMPART achieves the correct top-$k$ ranking with high probability under mild conditions, and demonstrate through extensive simulation studies that RAMPART consistently outperforms popular feature importance methods, concluding with a high-dimensional genomics case study.

Method: Introduces a novel biwhitening method inspired by Sinkhorn-Knopp algorithm to stabilize variance across genes and cells. Uses RMT-based criterion to automatically select sparsity level for sparse PCA algorithms.

Result: The method systematically improves principal subspace reconstruction across seven single-cell RNA-seq technologies and consistently outperforms PCA, autoencoder, and diffusion-based methods in cell-type classification tasks.

Conclusion: The mathematically grounded approach retains PCA’s interpretability while enabling robust, hands-off inference of sparse principal components, making it suitable for diverse single-cell datasets.

Abstract: Single-cell RNA-seq provides detailed molecular snapshots of individual cells but is notoriously noisy. Variability stems from biological differences, PCR amplification bias, limited sequencing depth, and low capture efficiency, making it challenging to adapt computational pipelines to heterogeneous datasets or evolving technologies. As a result, most studies still rely on principal component analysis (PCA) for dimensionality reduction, valued for its interpretability and robustness. Here, we improve upon PCA with a Random Matrix Theory (RMT)-based approach that guides the inference of sparse principal components using existing sparse PCA algorithms. We first introduce a novel biwhitening method, inspired by the Sinkhorn-Knopp algorithm, that simultaneously stabilizes variance across genes and cells. This enables the use of an RMT-based criterion to automatically select the sparsity level, rendering sparse PCA nearly parameter-free. Our mathematically grounded approach retains the interpretability of PCA while enabling robust, hands-off inference of sparse principal components. Across seven single-cell RNA-seq technologies and four sparse PCA algorithms, we show that this method systematically improves the reconstruction of the principal subspace and consistently outperforms PCA-, autoencoder-, and diffusion-based methods in cell-type classification tasks.

Method: Representing piecewise linear activation functions with tropical arithmetic, developing algorithms to compute regions where neural networks act as linear maps, and conducting computational experiments.

Result: The computational experiments provide insights into neural network training difficulties, particularly addressing overfitting problems and demonstrating the advantages of skip connections.

Conclusion: Tropical geometry offers a valuable mathematical framework for analyzing neural networks, with practical implications for understanding training challenges and architectural benefits like skip connections.

Abstract: Neural networks are important tools in machine learning. Representing piecewise linear activation functions with tropical arithmetic enables the application of tropical geometry. Algorithms are presented to compute regions where the neural networks are linear maps. Through computational experiments, we provide insights on the difficulty to train neural networks, in particular on the problems of overfitting and on the benefits of skip connections.

Method: Proposes a mathematical construct for multi-modal multi-scale data representation, derives neural attention mechanics from entropy minimization first principles, and develops an efficient dynamic programming algorithm for computation.

Result: The derived formulation is optimal (closest to standard Softmax attention while incorporating hierarchical biases). Enables training transformers from scratch in hierarchical settings and post-training hierarchical injection into pre-trained models.

Conclusion: The proposed hierarchical attention mechanism provides a mathematically principled solution for multi-scale, multi-modal transformer applications, offering both training flexibility and efficiency improvements in zero-shot scenarios.

Abstract: Transformers and their attention mechanism have been revolutionary in the field of Machine Learning. While originally proposed for the language data, they quickly found their way to the image, video, graph, etc. data modalities with various signal geometries. Despite this versatility, generalizing the attention mechanism to scenarios where data is presented at different scales from potentially different modalities is not straightforward. The attempts to incorporate hierarchy and multi-modality within transformers are largely based on ad hoc heuristics, which are not seamlessly generalizable to similar problems with potentially different structures. To address this problem, in this paper, we take a fundamentally different approach: we first propose a mathematical construct to represent multi-modal, multi-scale data. We then mathematically derive the neural attention mechanics for the proposed construct from the first principle of entropy minimization. We show that the derived formulation is optimal in the sense of being the closest to the standard Softmax attention while incorporating the inductive biases originating from the hierarchical/geometric information of the problem. We further propose an efficient algorithm based on dynamic programming to compute our derived attention mechanism. By incorporating it within transformers, we show that the proposed hierarchical attention mechanism not only can be employed to train transformer models in hierarchical/multi-modal settings from scratch, but it can also be used to inject hierarchical information into classical, pre-trained transformer models post training, resulting in more efficient models in zero-shot manner.

Method: EmoHeal uses a fine-tuned XLM-RoBERTa model to detect 27 fine-grained emotions from user text, maps them to musical parameters via a knowledge graph based on music therapy principles (GEMS, iso-principle), and retrieves audiovisual content using CLAMP3 model to guide users through a three-stage “match-guide-target” process.

Result: A within-subjects study (N=40) showed significant mood improvement (M=4.12, p<0.001) and high perceived emotion recognition accuracy (M=4.05, p<0.001), with strong correlation between perceived accuracy and therapeutic outcome (r=0.72, p<0.001).

Conclusion: The findings establish the viability of theory-driven, emotion-aware digital wellness tools and provide a scalable AI blueprint for operationalizing music therapy principles.

Abstract: Existing digital mental wellness tools often overlook the nuanced emotional states underlying everyday challenges. For example, pre-sleep anxiety affects more than 1.5 billion people worldwide, yet current approaches remain largely static and “one-size-fits-all”, failing to adapt to individual needs. In this work, we present EmoHeal, an end-to-end system that delivers personalized, three-stage supportive narratives. EmoHeal detects 27 fine-grained emotions from user text with a fine-tuned XLM-RoBERTa model, mapping them to musical parameters via a knowledge graph grounded in music therapy principles (GEMS, iso-principle). EmoHeal retrieves audiovisual content using the CLAMP3 model to guide users from their current state toward a calmer one (“match-guide-target”). A within-subjects study (N=40) demonstrated significant supportive effects, with participants reporting substantial mood improvement (M=4.12, p<0.001) and high perceived emotion recognition accuracy (M=4.05, p<0.001). A strong correlation between perceived accuracy and therapeutic outcome (r=0.72, p<0.001) validates our fine-grained approach. These findings establish the viability of theory-driven, emotion-aware digital wellness tools and provides a scalable AI blueprint for operationalizing music therapy principles.

Method: Two innovations: 1) Frame quantization as cooperative game with SPQE for efficient Shapley estimates of layer sensitivities and interactions; 2) IMPQ translates Shapley estimates into binary quadratic optimization to assign 2 or 4-bit precision under memory constraints.

Result: IMPQ demonstrates superior performance across Llama-3, Gemma-2, and Qwen-3 models on three PTQ backends, cutting Perplexity by 20-80% relative to best baselines, with margin increasing as bit-width tightens from 4 to 2 bits.

Conclusion: IMPQ provides a scalable and effective solution for low-precision quantization by properly accounting for inter-layer interactions, enabling better deployment of LLMs on resource-constrained devices.

Abstract: Large Language Models (LLMs) promise impressive capabilities, yet their multi-billion-parameter scale makes on-device or low-resource deployment prohibitive. Mixed-precision quantization offers a compelling solution, but existing methods struggle when the average precision drops below four bits, as they rely on isolated, layer-specific metrics that overlook critical inter-layer interactions affecting overall performance. In this paper, we propose two innovations to address these limitations. First, we frame the mixed-precision quantization problem as a cooperative game among layers and introduce Shapley-based Progressive Quantization Estimation (SPQE) to efficiently obtain accurate Shapley estimates of layer sensitivities and inter-layer interactions. Second, building upon SPQE, we propose Interaction-aware Mixed-Precision Quantization (IMPQ) which translates these Shapley estimates into a binary quadratic optimization formulation, assigning either 2 or 4-bit precision to layers under strict memory constraints. Comprehensive experiments conducted on Llama-3, Gemma-2, and Qwen-3 models across three independent PTQ backends (Quanto, HQQ, GPTQ) demonstrate IMPQ’s scalability and consistently superior performance compared to methods relying solely on isolated metrics. Across average precisions spanning 4 bit down to 2 bit, IMPQ cuts Perplexity by 20 to 80 percent relative to the best baseline, with the margin growing as the bit-width tightens.

Method: Proposes EvoReasoner (temporal-aware multi-hop reasoning with global-local entity grounding, multi-route decomposition, and temporally grounded scoring) and EvoKG (noise-tolerant KG evolution module with confidence-based contradiction resolution and temporal trend tracking).

Result: Outperforms both prompting-based and KG-enhanced baselines on temporal QA benchmarks, narrowing gap between small and large LLMs. An 8B-parameter model using this approach matches performance of a 671B model prompted seven months later.

Conclusion: Combining temporal reasoning with KG evolution is crucial for robust and up-to-date LLM performance in dynamic question answering.

Abstract: Large language models (LLMs) excel at many language understanding tasks but struggle to reason over knowledge that evolves. To address this, recent work has explored augmenting LLMs with knowledge graphs (KGs) to provide structured, up-to-date information. However, many existing approaches assume a static snapshot of the KG and overlook the temporal dynamics and factual inconsistencies inherent in real-world data. To address the challenge of reasoning over temporally shifting knowledge, we propose EvoReasoner, a temporal-aware multi-hop reasoning algorithm that performs global-local entity grounding, multi-route decomposition, and temporally grounded scoring. To ensure that the underlying KG remains accurate and up-to-date, we introduce EvoKG, a noise-tolerant KG evolution module that incrementally updates the KG from unstructured documents through confidence-based contradiction resolution and temporal trend tracking. We evaluate our approach on temporal QA benchmarks and a novel end-to-end setting where the KG is dynamically updated from raw documents. Our method outperforms both prompting-based and KG-enhanced baselines, effectively narrowing the gap between small and large LLMs on dynamic question answering. Notably, an 8B-parameter model using our approach matches the performance of a 671B model prompted seven months later. These results highlight the importance of combining temporal reasoning with KG evolution for robust and up-to-date LLM performance. Our code is publicly available at github.com/junhongmit/TREK.

Method: SolarCAST models three classes of confounding factors using neural components: (i) observable synchronous variables via embedding module, (ii) latent synchronous factors via spatio-temporal graph neural network, and (iii) time-lagged influences via gated transformer that learns temporal shifts.

Result: SolarCAST outperforms leading time-series and multimodal baselines across diverse geographical conditions and achieves a 25.9% error reduction over the top commercial forecaster Solcast.

Conclusion: SolarCAST offers a lightweight, practical, and generalizable solution for localized solar forecasting that requires only public sensor data, making it more accessible than hardware-dependent alternatives.

Abstract: Accurate solar forecasting underpins effective renewable energy management. We present SolarCAST, a causally informed model predicting future global horizontal irradiance (GHI) at a target site using only historical GHI from site X and nearby stations S - unlike prior work that relies on sky-camera or satellite imagery requiring specialized hardware and heavy preprocessing. To deliver high accuracy with only public sensor data, SolarCAST models three classes of confounding factors behind X-S correlations using scalable neural components: (i) observable synchronous variables (e.g., time of day, station identity), handled via an embedding module; (ii) latent synchronous factors (e.g., regional weather patterns), captured by a spatio-temporal graph neural network; and (iii) time-lagged influences (e.g., cloud movement across stations), modeled with a gated transformer that learns temporal shifts. It outperforms leading time-series and multimodal baselines across diverse geographical conditions, and achieves a 25.9% error reduction over the top commercial forecaster, Solcast. SolarCAST offers a lightweight, practical, and generalizable solution for localized solar forecasting.

Method: FRAUDGUESS spots new fraud types as micro-clusters in a designed feature space and uses visualization, heatmaps, and an interactive dashboard for justification.

Result: The system discovered three new fraudulent behaviors in a real million-scale financial dataset, with two confirmed as fraudulent/suspicious by experts, catching hundreds of previously unnoticed fraudulent transactions.

Conclusion: FRAUDGUESS effectively detects unknown fraud types and provides justification evidence, showing practical value with real-world deployment consideration in a financial institution.

Abstract: Given a set of financial transactions (who buys from whom, when, and for how much), as well as prior information from buyers and sellers, how can we find fraudulent transactions? If we have labels for some transactions for known types of fraud, we can build a classifier. However, we also want to find new types of fraud, still unknown to the domain experts (‘Detection’). Moreover, we also want to provide evidence to experts that supports our opinion (‘Justification’). In this paper, we propose FRAUDGUESS, to achieve two goals: (a) for ‘Detection’, it spots new types of fraud as micro-clusters in a carefully designed feature space; (b) for ‘Justification’, it uses visualization and heatmaps for evidence, as well as an interactive dashboard for deep dives. FRAUDGUESS is used in real life and is currently considered for deployment in an Anonymous Financial Institution (AFI). Thus, we also present the three new behaviors that FRAUDGUESS discovered in a real, million-scale financial dataset. Two of these behaviors are deemed fraudulent or suspicious by domain experts, catching hundreds of fraudulent transactions that would otherwise go un-noticed.

Method: WIEN-INR employs a multi-scale architecture that distributes modeling across different resolution scales and uses a specialized kernel network at the finest scale to recover subtle details, allowing for smaller networks while retaining full information spectrum.

Result: Extensive experiments on diverse scientific datasets show that WIEN-INR achieves superior reconstruction fidelity while maintaining compact model size, preserving training efficiency and reducing storage costs.

Conclusion: WIEN-INR demonstrates practical neural representation framework for high-fidelity scientific data encoding, extending INR applicability to domains requiring efficient preservation of fine detail.

Abstract: Implicit neural representations (INRs) have emerged as a compact and parametric alternative to discrete array-based data representations, encoding information directly in neural network weights to enable resolution-independent representation and memory efficiency. However, existing INR approaches, when constrained to compact network sizes, struggle to faithfully represent the multi-scale structures, high-frequency information, and fine textures that characterize the majority of scientific datasets. To address this limitation, we propose WIEN-INR, a wavelet-informed implicit neural representation that distributes modeling across different resolution scales and employs a specialized kernel network at the finest scale to recover subtle details. This multi-scale architecture allows for the use of smaller networks to retain the full spectrum of information while preserving the training efficiency and reducing storage cost. Through extensive experiments on diverse scientific datasets spanning different scales and structural complexities, WIEN-INR achieves superior reconstruction fidelity while maintaining a compact model size. These results demonstrate WIEN-INR as a practical neural representation framework for high-fidelity scientific data encoding, extending the applicability of INRs to domains where efficient preservation of fine detail is essential.

Method: Proposes Experience-Weighted Attraction with Vector Quantization (EWA-VQ-ODT), which maintains per-action mental accounts using a vector-quantized codebook. Continuous actions are mapped to codes storing scalar attractions updated online through decay and reward-based reinforcement, which modulate attention by biasing action token columns.

Result: EWA-VQ-ODT improves sample efficiency and average return over standard ODT on continuous-control benchmarks, particularly during early training. The module is computationally efficient and interpretable.

Conclusion: The proposed EWA-VQ-ODT successfully enhances Online Decision Transformers by incorporating action-specific memory through vector quantization, providing better performance while maintaining computational efficiency and interpretability.

Abstract: Transformers have emerged as a compelling architecture for sequential decision-making by modeling trajectories via self-attention. In reinforcement learning (RL), they enable return-conditioned control without relying on value function approximation. Decision Transformers (DTs) exploit this by casting RL as supervised sequence modeling, but they are restricted to offline data and lack exploration. Online Decision Transformers (ODTs) address this limitation through entropy-regularized training on on-policy rollouts, offering a stable alternative to traditional RL methods like Soft Actor-Critic, which depend on bootstrapped targets and reward shaping. Despite these advantages, ODTs use standard attention, which lacks explicit memory of action-specific outcomes. This leads to inefficiencies in learning long-term action effectiveness. Inspired by cognitive models such as Experience-Weighted Attraction (EWA), we propose Experience-Weighted Attraction with Vector Quantization for Online Decision Transformers (EWA-VQ-ODT), a lightweight module that maintains per-action mental accounts summarizing recent successes and failures. Continuous actions are routed via direct grid lookup to a compact vector-quantized codebook, where each code stores a scalar attraction updated online through decay and reward-based reinforcement. These attractions modulate attention by biasing the columns associated with action tokens, requiring no change to the backbone or training objective. On standard continuous-control benchmarks, EWA-VQ-ODT improves sample efficiency and average return over ODT, particularly in early training. The module is computationally efficient, interpretable via per-code traces, and supported by theoretical guarantees that bound the attraction dynamics and its impact on attention drift.

Method: A policy gradient optimization method that leverages dual representation of coherent risk measures and extends the envelope theorem to continuous cases, with extensions to episodic settings.

Result: The algorithm achieves a convergence rate of O(T^{-1/2}+r^{-1/2}) where T is policy gradient iterations and r is sample size, with global convergence guarantees in episodic settings.

Conclusion: The proposed policy gradient method effectively handles MDPs with unknown parameters and coherent risk measures, providing theoretical convergence guarantees and practical applicability to episodic problems.

Abstract: Motivated by many application problems, we consider Markov decision processes (MDPs) with a general loss function and unknown parameters. To mitigate the epistemic uncertainty associated with unknown parameters, we take a Bayesian approach to estimate the parameters from data and impose a coherent risk functional (with respect to the Bayesian posterior distribution) on the loss. Since this formulation usually does not satisfy the interchangeability principle, it does not admit Bellman equations and cannot be solved by approaches based on dynamic programming. Therefore, We propose a policy gradient optimization method, leveraging the dual representation of coherent risk measures and extending the envelope theorem to continuous cases. We then show the stationary analysis of the algorithm with a convergence rate of $O(T^{-1/2}+r^{-1/2})$, where $T$ is the number of policy gradient iterations and $r$ is the sample size of the gradient estimator. We further extend our algorithm to an episodic setting, and establish the global convergence of the extended algorithm and provide bounds on the number of iterations needed to achieve an error bound $O(\epsilon)$ in each episode.

Method: Two-stage approach: 1) Probabilistic neural goal estimator predicts long-term targets, 2) Koopman operator-based refinement module incorporates intention and history into nonlinear feature space for linear prediction.

Result: Competitive accuracy across ETH/UCY, Waymo Open Motion Dataset, and nuScenes benchmarks, with mode-level interpretability and practical efficiency.

Conclusion: KoopCast successfully combines predictive accuracy with interpretability and efficiency by leveraging Koopman operator theory for linear representation of nonlinear dynamics in trajectory forecasting.

Abstract: We present KoopCast, a lightweight yet efficient model for trajectory forecasting in general dynamic environments. Our approach leverages Koopman operator theory, which enables a linear representation of nonlinear dynamics by lifting trajectories into a higher-dimensional space. The framework follows a two-stage design: first, a probabilistic neural goal estimator predicts plausible long-term targets, specifying where to go; second, a Koopman operator-based refinement module incorporates intention and history into a nonlinear feature space, enabling linear prediction that dictates how to go. This dual structure not only ensures strong predictive accuracy but also inherits the favorable properties of linear operators while faithfully capturing nonlinear dynamics. As a result, our model offers three key advantages: (i) competitive accuracy, (ii) interpretability grounded in Koopman spectral theory, and (iii) low-latency deployment. We validate these benefits on ETH/UCY, the Waymo Open Motion Dataset, and nuScenes, which feature rich multi-agent interactions and map-constrained nonlinear motion. Across benchmarks, KoopCast consistently delivers high predictive accuracy together with mode-level interpretability and practical efficiency.

Method: Proposed a composite reward function with specific penalties for reward hacking behaviors. Extended RLVR framework with this new reward model to enforce proper reasoning formats.

Result: Experiments showed the approach leads to better-formatted reasoning with less reward hacking while maintaining good accuracy compared to baselines.

Conclusion: This work represents progress toward reducing reward hacking and improving reliability of RLVR-based models in medical applications.

Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has recently shown that large language models (LLMs) can develop their own reasoning without direct supervision. However, applications in the medical domain, specifically for question answering, are susceptible to significant reward hacking during the reasoning phase. Our work addresses two primary forms of this behavior: i) providing a final answer without preceding reasoning, and ii) employing non-standard reasoning formats to exploit the reward mechanism. To mitigate these, we introduce a composite reward function with specific penalties for these behaviors. Our experiments show that extending RLVR with our proposed reward model leads to better-formatted reasoning with less reward hacking and good accuracy compared to the baselines. This approach marks a step toward reducing reward hacking and enhancing the reliability of models utilizing RLVR.

Method: Review theoretical foundations, present eight representative estimators, conduct controlled experiments analyzing factors like noise, curvature, and sample size, and compare estimators on synthetic and real-world datasets with principled hyperparameter tuning.

Result: The study provides practical guidance through systematic evaluation and finds that simpler dimension estimation methods tend to perform better for this general problem domain.

Conclusion: For manifold dimension estimation - a problem of general applicability - simpler methods are often more effective than complex approaches, and the survey offers comprehensive guidance for method selection and application.

Abstract: The manifold hypothesis suggests that high-dimensional data often lie on or near a low-dimensional manifold. Estimating the dimension of this manifold is essential for leveraging its structure, yet existing work on dimension estimation is fragmented and lacks systematic evaluation. This article provides a comprehensive survey for both researchers and practitioners. We review often-overlooked theoretical foundations and present eight representative estimators. Through controlled experiments, we analyze how individual factors such as noise, curvature, and sample size affect performance. We also compare the estimators on diverse synthetic and real-world datasets, introducing a principled approach to dataset-specific hyperparameter tuning. Our results offer practical guidance and suggest that, for a problem of this generality, simpler methods often perform better.

Method: DAC formalizes each agent’s local task as a Contextual MDP, models step-wise dynamics using latent variables as contexts, introduces context-based value functions, and derives optimistic marginal values to promote cooperative actions.

Result: Experimental evaluation on matrix games, predator-prey, and SMAC tasks shows DAC achieves superior performance against multiple baselines.

Conclusion: DAC effectively addresses both non-stationarity and relative overgeneralization in fully decentralized cooperative MARL through its dynamics-aware context modeling approach.

Abstract: This paper studies fully decentralized cooperative multi-agent reinforcement learning, where each agent solely observes the states, its local actions, and the shared rewards. The inability to access other agents’ actions often leads to non-stationarity during value function updates and relative overgeneralization during value function estimation, hindering effective cooperative policy learning. However, existing works fail to address both issues simultaneously, due to their inability to model the joint policy of other agents in a fully decentralized setting. To overcome this limitation, we propose a novel method named Dynamics-Aware Context (DAC), which formalizes the task, as locally perceived by each agent, as an Contextual Markov Decision Process, and further addresses both non-stationarity and relative overgeneralization through dynamics-aware context modeling. Specifically, DAC attributes the non-stationary local task dynamics of each agent to switches between unobserved contexts, each corresponding to a distinct joint policy. Then, DAC models the step-wise dynamics distribution using latent variables and refers to them as contexts. For each agent, DAC introduces a context-based value function to address the non-stationarity issue during value function update. For value function estimation, an optimistic marginal value is derived to promote the selection of cooperative actions, thereby addressing the relative overgeneralization issue. Experimentally, we evaluate DAC on various cooperative tasks (including matrix game, predator and prey, and SMAC), and its superior performance against multiple baselines validates its effectiveness.

Method: Combines the expressiveness of normalizing flows for density estimation with the analytical tractability of Bernstein polynomials to create a class of models that can approximate general nonlinear stochastic dynamics.

Result: Empirical results demonstrate the model’s efficacy over state-of-the-art data-driven methods for belief propagation, particularly for highly nonlinear systems with non-additive, non-Gaussian noise.

Conclusion: The paper establishes theoretical foundations for models that achieve both universal approximation of nonlinear stochastic dynamics and analytical belief propagation capability.

Abstract: Predicting the distribution of future states in a stochastic system, known as belief propagation, is fundamental to reasoning under uncertainty. However, nonlinear dynamics often make analytical belief propagation intractable, requiring approximate methods. When the system model is unknown and must be learned from data, a key question arises: can we learn a model that (i) universally approximates general nonlinear stochastic dynamics, and (ii) supports analytical belief propagation? This paper establishes the theoretical foundations for a class of models that satisfy both properties. The proposed approach combines the expressiveness of normalizing flows for density estimation with the analytical tractability of Bernstein polynomials. Empirical results show the efficacy of our learned model over state-of-the-art data-driven methods for belief propagation, especially for highly non-linear systems with non-additive, non-Gaussian noise.

Method: Developed a normalized stochastic variance-reduced bilevel gradient descent algorithm that avoids clipping operations and handles interdependent gradient sequences under heavy-tailed noise conditions.

Result: Established convergence rate for nonconvex decentralized bilevel optimization under heavy-tailed noises, with experimental results confirming the algorithm’s effectiveness.

Conclusion: This is the first decentralized bilevel optimization algorithm with rigorous theoretical guarantees under heavy-tailed noise conditions, addressing limitations of existing methods.

Abstract: Existing decentralized stochastic optimization methods assume the lower-level loss function is strongly convex and the stochastic gradient noise has finite variance. These strong assumptions typically are not satisfied in real-world machine learning models. To address these limitations, we develop a novel decentralized stochastic bilevel optimization algorithm for the nonconvex bilevel optimization problem under heavy-tailed noises. Specifically, we develop a normalized stochastic variance-reduced bilevel gradient descent algorithm, which does not rely on any clipping operation. Moreover, we establish its convergence rate by innovatively bounding interdependent gradient sequences under heavy-tailed noises for nonconvex decentralized bilevel optimization problems. As far as we know, this is the first decentralized bilevel optimization algorithm with rigorous theoretical guarantees under heavy-tailed noises. The extensive experimental results confirm the effectiveness of our algorithm in handling heavy-tailed noises.

Method: PolyJuice identifies the direction of distribution shift between correctly and incorrectly classified samples through lightweight offline black-box access, then universally steers all generated images towards SID’s failure modes. It also enables efficient resolution scaling through interpolation.

Result: PolyJuice-steered T2I models deceive SIDs up to 84% more effectively than unsteered counterparts. Tuning SIDs on PolyJuice-augmented datasets improves detector performance by up to 30%.

Conclusion: PolyJuice provides an effective black-box red-teaming solution that both identifies SID vulnerabilities and enhances detector robustness through adversarial training.

Abstract: Synthetic image detectors (SIDs) are a key defense against the risks posed by the growing realism of images from text-to-image (T2I) models. Red teaming improves SID’s effectiveness by identifying and exploiting their failure modes via misclassified synthetic images. However, existing red-teaming solutions (i) require white-box access to SIDs, which is infeasible for proprietary state-of-the-art detectors, and (ii) generate image-specific attacks through expensive online optimization. To address these limitations, we propose PolyJuice, the first black-box, image-agnostic red-teaming method for SIDs, based on an observed distribution shift in the T2I latent space between samples correctly and incorrectly classified by the SID. PolyJuice generates attacks by (i) identifying the direction of this shift through a lightweight offline process that only requires black-box access to the SID, and (ii) exploiting this direction by universally steering all generated images towards the SID’s failure modes. PolyJuice-steered T2I models are significantly more effective at deceiving SIDs (up to 84%) compared to their unsteered counterparts. We also show that the steering directions can be estimated efficiently at lower resolutions and transferred to higher resolutions using simple interpolation, reducing computational overhead. Finally, tuning SID models on PolyJuice-augmented datasets notably enhances the performance of the detectors (up to 30%).

Method: The authors analyze two aggregation methods: simple averaging (ZO-Avg) and a new Projection Alignment method (ZO-Align) derived from local surrogate minimization. They derive convergence rates across strongly convex, convex, non-convex, and stochastic settings.

Result: A stark dichotomy emerges: ZO-Avg is always query-inefficient with more than one query per iteration (single-query optimal), while ZO-Align performs better with more queries (full-subspace estimation optimal). The multi-query problem reduces to choosing between these two classic algorithms.

Conclusion: The optimal query allocation strategy is entirely dictated by the aggregation method used, not by intermediate query sizes. Theoretical findings are consistently validated through extensive experiments, clarifying the fundamental trade-off in zeroth-order optimization.

Abstract: Zeroth-order (ZO) optimization provides a powerful framework for problems where explicit gradients are unavailable and have to be approximated using only queries to function value. The prevalent single-query approach is simple, but suffers from high estimation variance, motivating a multi-query paradigm to improves estimation accuracy. This, however, creates a critical trade-off: under a fixed budget of queries (i.e. cost), queries per iteration and the total number of optimization iterations are inversely proportional to one another. How to best allocate this budget is a fundamental, under-explored question. This work systematically resolves this query allocation problem. We analyze two aggregation methods: the de facto simple averaging (ZO-Avg), and a new Projection Alignment method (ZO-Align) we derive from local surrogate minimization. By deriving convergence rates for both methods that make the dependence on the number of queries explicit across strongly convex, convex, non-convex, and stochastic settings, we uncover a stark dichotomy: For ZO-Avg, we prove that using more than one query per iteration is always query-inefficient, rendering the single-query approach optimal. On the contrary, ZO-Align generally performs better with more queries per iteration, resulting in a full-subspace estimation as the optimal approach. Thus, our work clarifies that the multi-query problem boils down to a choice not about an intermediate query size, but between two classic algorithms, a choice dictated entirely by the aggregation method used. These theoretical findings are also consistently validated by extensive experiments.

Method: Creates shared Gaussian latent space where each data type (images, text, labels) has dedicated encoder/decoder pairs mapping to disjoint zones. Tasks are expressed as compositions of these components.

Result: LZN improves FID on CIFAR10 from 2.76 to 2.59 when combined with Rectified Flow; outperforms MoCo and SimCLR by 9.3% and 0.2% on ImageNet linear classification; achieves SOTA classification accuracy on CIFAR10 with joint generation.

Conclusion: LZN demonstrates promise as a unified principle for multiple ML tasks, showing improvements across generation, representation learning, and classification without requiring task-specific modifications.

Abstract: Generative modeling, representation learning, and classification are three core problems in machine learning (ML), yet their state-of-the-art (SoTA) solutions remain largely disjoint. In this paper, we ask: Can a unified principle address all three? Such unification could simplify ML pipelines and foster greater synergy across tasks. We introduce Latent Zoning Network (LZN) as a step toward this goal. At its core, LZN creates a shared Gaussian latent space that encodes information across all tasks. Each data type (e.g., images, text, labels) is equipped with an encoder that maps samples to disjoint latent zones, and a decoder that maps latents back to data. ML tasks are expressed as compositions of these encoders and decoders: for example, label-conditional image generation uses a label encoder and image decoder; image embedding uses an image encoder; classification uses an image encoder and label decoder. We demonstrate the promise of LZN in three increasingly complex scenarios: (1) LZN can enhance existing models (image generation): When combined with the SoTA Rectified Flow model, LZN improves FID on CIFAR10 from 2.76 to 2.59-without modifying the training objective. (2) LZN can solve tasks independently (representation learning): LZN can implement unsupervised representation learning without auxiliary loss functions, outperforming the seminal MoCo and SimCLR methods by 9.3% and 0.2%, respectively, on downstream linear classification on ImageNet. (3) LZN can solve multiple tasks simultaneously (joint generation and classification): With image and label encoders/decoders, LZN performs both tasks jointly by design, improving FID and achieving SoTA classification accuracy on CIFAR10. The code and trained models are available at https://github.com/microsoft/latent-zoning-networks. The project website is at https://zinanlin.me/blogs/latent_zoning_networks.html.

Method: Proposes an Expert Block Framework with a Trajectory Context Summarizer (TCS) - a deterministic block that transforms raw training trajectories into structured context. This enables small LLMs to analyze optimization progress reliably. Tested with phi4:reasoning14B and qwen2.5-coder:32B using a 10-trial budget.

Result: The TCS-enabled HPT pipeline achieves average performance within ~0.9 percentage points of GPT-4 across six diverse tasks, demonstrating that small LLMs can match large model performance when properly contextualized.

Conclusion: Small LLMs can effectively perform hyper-parameter tuning when equipped with appropriate context summarization techniques, making HPT more accessible and computationally efficient without sacrificing performance.

Abstract: Hyper-parameter Tuning (HPT) is a necessary step in machine learning (ML) pipelines but becomes computationally expensive and opaque with larger models. Recently, Large Language Models (LLMs) have been explored for HPT, yet most rely on models exceeding 100 billion parameters. We propose an Expert Block Framework for HPT using Small LLMs. At its core is the Trajectory Context Summarizer (TCS), a deterministic block that transforms raw training trajectories into structured context, enabling small LLMs to analyze optimization progress with reliability comparable to larger models. Using two locally-run LLMs (phi4:reasoning14B and qwen2.5-coder:32B) and a 10-trial budget, our TCS-enabled HPT pipeline achieves average performance within ~0.9 percentage points of GPT-4 across six diverse tasks.

Method: Uses the natural-gradient descent algorithm called the Bayesian Learning Rule (BLR) by Khan and Rue (2023) to analyze VB solutions and demonstrate consequences including simplification of Bayes’ rule as natural gradient addition.

Result: Shows that VB solutions require natural gradients under certain conditions, demonstrates simplification of Bayes’ rule, generalization of quadratic surrogates, and enables large-scale VB implementations for large language models.

Conclusion: While the connection between information geometry and Bayes is not new, the paper emphasizes their common origins to encourage more interdisciplinary work between these fields, with practical applications in machine learning.

Abstract: We highlight a fundamental connection between information geometry and variational Bayes (VB) and discuss its consequences for machine learning. Under certain conditions, a VB solution always requires estimation or computation of natural gradients. We show several consequences of this fact by using the natural-gradient descent algorithm of Khan and Rue (2023) called the Bayesian Learning Rule (BLR). These include (i) a simplification of Bayes’ rule as addition of natural gradients, (ii) a generalization of quadratic surrogates used in gradient-based methods, and (iii) a large-scale implementation of VB algorithms for large language models. Neither the connection nor its consequences are new but we further emphasize the common origins of the two fields of information geometry and Bayes with a hope to facilitate more work at the intersection of the two fields.

Method: A simple controlled experimental framework using the dSprites dataset with procedurally generated modifying factors to investigate influences on class discovery success.

Result: Empirical results show that the benefit of the number of known classes reaches a saturation point beyond which discovery performance plateaus, with diminishing returns across different settings.

Conclusion: The findings provide insights for cost-benefit analysis for practitioners and serve as a starting point for more rigorous future research on class discovery in complex real-world datasets.

Abstract: Novel class discovery is essential for ML models to adapt to evolving real-world data, with applications ranging from scientific discovery to robotics. However, these datasets contain complex and entangled factors of variation, making a systematic study of class discovery difficult. As a result, many fundamental questions are yet to be answered on why and when new class discoveries are more likely to be successful. To address this, we propose a simple controlled experimental framework using the dSprites dataset with procedurally generated modifying factors. This allows us to investigate what influences successful class discovery. In particular, we study the relationship between the number of known/unknown classes and discovery performance, as well as the impact of known class ‘coverage’ on discovering new classes. Our empirical results indicate that the benefit of the number of known classes reaches a saturation point beyond which discovery performance plateaus. The pattern of diminishing return across different settings provides an insight for cost-benefit analysis for practitioners and a starting point for more rigorous future research of class discovery on complex real-world datasets.

Method: Leverages dropout as a gradient compression mechanism to reduce the computational and memory overhead in influence function computation, while preserving critical components of data influence through theoretical guarantees.

Result: The method significantly reduces computational and memory costs during both influence function computation and gradient compression process, enabling application to modern large-scale models while maintaining influence accuracy.

Conclusion: Dropout-based gradient compression provides an efficient and practical solution for computing influence functions on large-scale models, enhancing model transparency and data selection capabilities without prohibitive resource requirements.

Abstract: Assessing the impact the training data on machine learning models is crucial for understanding the behavior of the model, enhancing the transparency, and selecting training data. Influence function provides a theoretical framework for quantifying the effect of training data points on model’s performance given a specific test data. However, the computational and memory costs of influence function presents significant challenges, especially for large-scale models, even when using approximation methods, since the gradients involved in computation are as large as the model itself. In this work, we introduce a novel approach that leverages dropout as a gradient compression mechanism to compute the influence function more efficiently. Our method significantly reduces computational and memory overhead, not only during the influence function computation but also in gradient compression process. Through theoretical analysis and empirical validation, we demonstrate that our method could preserves critical components of the data influence and enables its application to modern large-scale models.

Method: The paper introduces a personalized prediction approach where an interpretable sparse linear classifier is learned per query point for specific sub-populations represented by halfspaces. It provides distribution-specific PAC-learning algorithms and combines reference-class learning with sparse linear representation learning.

Result: The authors prove an upper bound of O(opt^{1/4}) for personalized prediction with sparse linear classifiers and homogeneous halfspace subsets, and demonstrate performance on standard benchmark datasets.

Conclusion: The work establishes PAC-learnability foundations for personalized prediction with interpretable sparse linear models, providing theoretical guarantees and practical validation.

Abstract: In machine learning applications, predictive models are trained to serve future queries across the entire data distribution. Real-world data often demands excessively complex models to achieve competitive performance, however, sacrificing interpretability. Hence, the growing deployment of machine learning models in high-stakes applications, such as healthcare, motivates the search for methods for accurate and explainable predictions. This work proposes a Personalized Prediction scheme, where an easy-to-interpret predictor is learned per query. In particular, we wish to produce a “sparse linear” classifier with competitive performance specifically on some sub-population that includes the query point. The goal of this work is to study the PAC-learnability of this prediction model for sub-populations represented by “halfspaces” in a label-agnostic setting. We first give a distribution-specific PAC-learning algorithm for learning reference classes for personalized prediction. By leveraging both the reference-class learning algorithm and a list learner of sparse linear representations, we prove the first upper bound, $O(\mathrm{opt}^{1/4} )$, for personalized prediction with sparse linear classifiers and homogeneous halfspace subsets. We also evaluate our algorithms on a variety of standard benchmark data sets.

Method: Used Cornell Newsroom dataset (1.3M article-summary pairs) with BERT embeddings to transform text into numerical representations. Framed as binary classification problem, testing logistic regression, feed-forward neural networks, and LSTM networks.

Result: LSTM networks outperformed baseline methods (Lede-3 and simpler models) in F1 score and ROUGE-1 metrics due to their ability to capture sequential dependencies.

Conclusion: Automated summarization has significant potential for improving content management systems in online news platforms, enabling more efficient content organization and enhanced user experiences.

Abstract: In the age of information overload, content management for online news articles relies on efficient summarization to enhance accessibility and user engagement. This article addresses the challenge of extractive text summarization by employing advanced machine learning techniques to generate concise and coherent summaries while preserving the original meaning. Using the Cornell Newsroom dataset, comprising 1.3 million article-summary pairs, we developed a pipeline leveraging BERT embeddings to transform textual data into numerical representations. By framing the task as a binary classification problem, we explored various models, including logistic regression, feed-forward neural networks, and long short-term memory (LSTM) networks. Our findings demonstrate that LSTM networks, with their ability to capture sequential dependencies, outperform baseline methods like Lede-3 and simpler models in F1 score and ROUGE-1 metrics. This study underscores the potential of automated summarization in improving content management systems for online news platforms, enabling more efficient content organization and enhanced user experiences.

Method: Proposes H2T2, an online two-threshold hierarchical inference policy that continuously adapts thresholds on local model confidence scores. The thresholds determine local prediction and offloading decisions. For calibrated models, a closed-form solution is provided; for uncalibrated models, H2T2 achieves sublinear regret with no training required.

Result: Simulations on real-world datasets show H2T2 consistently outperforms naive and single-threshold policies, sometimes even surpassing offline optima. The policy demonstrates robustness to distribution shifts and adapts effectively to mismatched classifiers.

Conclusion: H2T2 provides an effective model-agnostic solution for hierarchical inference systems that learns during inference with limited feedback, offering significant performance improvements over baseline approaches while maintaining adaptability to changing conditions.

Abstract: We focus on a binary classification problem in an edge intelligence system where false negatives are more costly than false positives. The system has a compact, locally deployed model, which is supplemented by a larger, remote model, which is accessible via the network by incurring an offloading cost. For each sample, our system first uses the locally deployed model for inference. Based on the output of the local model, the sample may be offloaded to the remote model. This work aims to understand the fundamental trade-off between classification accuracy and these offloading costs within such a hierarchical inference (HI) system. To optimize this system, we propose an online learning framework that continuously adapts a pair of thresholds on the local model’s confidence scores. These thresholds determine the prediction of the local model and whether a sample is classified locally or offloaded to the remote model. We present a closed-form solution for the setting where the local model is calibrated. For the more general case of uncalibrated models, we introduce H2T2, an online two-threshold hierarchical inference policy, and prove it achieves sublinear regret. H2T2 is model-agnostic, requires no training, and learns in the inference phase using limited feedback. Simulations on real-world datasets show that H2T2 consistently outperforms naive and single-threshold HI policies, sometimes even surpassing offline optima. The policy also demonstrates robustness to distribution shifts and adapts effectively to mismatched classifiers.

Method: Extends policy evaluation within LSTD framework using Bellman-residual objective regularized with nonconvex projected minimax concave (PMC) penalty, solved via forward-reflected-backward splitting (FRBS) algorithm.

Result: Numerical experiments show the proposed approach substantially outperforms state-of-the-art feature-selection methods, especially with many noisy features.

Conclusion: The proposed PMC-regularized LSTD framework with FRBS algorithm provides an effective solution for feature selection in RL with theoretical convergence guarantees and superior performance in noisy environments.

Abstract: This work proposes an efficient batch algorithm for feature selection in reinforcement learning (RL) with theoretical convergence guarantees. To mitigate the estimation bias inherent in conventional regularization schemes, the first contribution extends policy evaluation within the classical least-squares temporal-difference (LSTD) framework by formulating a Bellman-residual objective regularized with the sparsity-inducing, nonconvex projected minimax concave (PMC) penalty. Owing to the weak convexity of the PMC penalty, this formulation can be interpreted as a special instance of a general nonmonotone-inclusion problem. The second contribution establishes novel convergence conditions for the forward-reflected-backward splitting (FRBS) algorithm to solve this class of problems. Numerical experiments on benchmark datasets demonstrate that the proposed approach substantially outperforms state-of-the-art feature-selection methods, particularly in scenarios with many noisy features.

Method: The authors model LLMs as linear functions over input embeddings and formulate example selection as an optimization problem. They derive a submodular surrogate objective enabling greedy algorithms with approximation guarantees, incorporating kernel tricks for high-dimensional spaces and optimal design-based regularization for diversity.

Result: Empirical evaluation shows significant improvements over standard retrieval methods across various classification tasks, demonstrating the benefits of structure-aware, diverse example selection.

Conclusion: The proposed principled approach to example selection in ICL outperforms traditional methods, particularly in real-world label-scarce scenarios, by optimizing for query-specific prediction accuracy rather than general generalization.

Abstract: In-context learning (ICL) has emerged as a powerful paradigm for adapting large language models (LLMs) to new and data-scarce tasks using only a few carefully selected task-specific examples presented in the prompt. However, given the limited context size of LLMs, a fundamental question arises: Which examples should be selected to maximize performance on a given user query? While nearest-neighbor-based methods like KATE have been widely adopted for this purpose, they suffer from well-known drawbacks in high-dimensional embedding spaces, including poor generalization and a lack of diversity. In this work, we study this problem of example selection in ICL from a principled, information theory-driven perspective. We first model an LLM as a linear function over input embeddings and frame the example selection task as a query-specific optimization problem: selecting a subset of exemplars from a larger example bank that minimizes the prediction error on a specific query. This formulation departs from traditional generalization-focused learning theoretic approaches by targeting accurate prediction for a specific query instance. We derive a principled surrogate objective that is approximately submodular, enabling the use of a greedy algorithm with an approximation guarantee. We further enhance our method by (i) incorporating the kernel trick to operate in high-dimensional feature spaces without explicit mappings, and (ii) introducing an optimal design-based regularizer to encourage diversity in the selected examples. Empirically, we demonstrate significant improvements over standard retrieval methods across a suite of classification tasks, highlighting the benefits of structure-aware, diverse example selection for ICL in real-world, label-scarce scenarios.

Method: RMT-KD leverages Random Matrix Theory for knowledge distillation, using spectral properties of hidden representations to identify and preserve informative directions. It applies RMT-based causal reduction layer by layer with self-distillation to maintain stability and accuracy.

Result: On GLUE, AG News, and CIFAR-10 datasets, RMT-KD achieves up to 80% parameter reduction with only 2% accuracy loss, delivering 2.8x faster inference and nearly halved power consumption.

Conclusion: RMT-KD establishes itself as a mathematically grounded approach to network distillation that effectively balances compression and performance.

Abstract: Large deep learning models such as BERT and ResNet achieve state-of-the-art performance but are costly to deploy at the edge due to their size and compute demands. We present RMT-KD, a compression method that leverages Random Matrix Theory (RMT) for knowledge distillation to iteratively reduce network size. Instead of pruning or heuristic rank selection, RMT-KD preserves only informative directions identified via the spectral properties of hidden representations. RMT-based causal reduction is applied layer by layer with self-distillation to maintain stability and accuracy. On GLUE, AG News, and CIFAR-10, RMT-KD achieves up to 80% parameter reduction with only 2% accuracy loss, delivering 2.8x faster inference and nearly halved power consumption. These results establish RMT-KD as a mathematically grounded approach to network distillation.

Method: Uses streaming covariance-spectrum statistics (entropy, eigenvalue gaps, KL divergence) from hidden activations, fed into a lightweight recurrent classifier to track temporal shifts in representation structure.

Result: EigenTrack provides interpretable real-time detection of hallucination and OOD drift using only a single forward pass without resampling.

Conclusion: EigenTrack offers a novel approach that preserves temporal context, aggregates global signals, and provides interpretable accuracy-latency trade-offs compared to existing black-box, grey-box, and white-box detection methods.

Abstract: Large language models (LLMs) offer broad utility but remain prone to hallucination and out-of-distribution (OOD) errors. We propose EigenTrack, an interpretable real-time detector that uses the spectral geometry of hidden activations, a compact global signature of model dynamics. By streaming covariance-spectrum statistics such as entropy, eigenvalue gaps, and KL divergence from random baselines into a lightweight recurrent classifier, EigenTrack tracks temporal shifts in representation structure that signal hallucination and OOD drift before surface errors appear. Unlike black- and grey-box methods, it needs only a single forward pass without resampling. Unlike existing white-box detectors, it preserves temporal context, aggregates global signals, and offers interpretable accuracy-latency trade-offs.

Method: The study evaluates classical physics-based models, empirical correction coefficients, and data-driven neural network architectures using a dataset of ~19,000 flights from nine A320-214 airframes with varying service ages. Age is incorporated either as an input feature or explicit multiplicative bias.

Result: Age-dependent correction factors and neural models substantially reduce bias and improve prediction accuracy compared to baseline models that consistently underestimate fuel consumption for older aircraft.

Conclusion: Accounting for ageing effects is essential in parametric and machine learning frameworks to improve reliability of operational and environmental assessments, but limitations exist due to small airframe sample size and lack of detailed maintenance records, highlighting the need for more diverse datasets.

Abstract: Accurate modelling of aircraft fuel-flow is crucial for both operational planning and environmental impact assessment, yet standard parametric models often neglect performance deterioration that occurs as aircraft age. This paper investigates multiple approaches to integrate engine ageing effects into fuel-flow prediction for the Airbus A320-214, using a comprehensive dataset of approximately nineteen thousand Quick Access Recorder flights from nine distinct airframes with varying years in service. We systematically evaluate classical physics-based models, empirical correction coefficients, and data-driven neural network architectures that incorporate age either as an input feature or as an explicit multiplicative bias. Results demonstrate that while baseline models consistently underestimate fuel consumption for older aircraft, the use of age-dependent correction factors and neural models substantially reduces bias and improves prediction accuracy. Nevertheless, limitations arise from the small number of airframes and the lack of detailed maintenance event records, which constrain the representativeness and generalization of age-based corrections. This study emphasizes the importance of accounting for the effects of ageing in parametric and machine learning frameworks to improve the reliability of operational and environmental assessments. The study also highlights the need for more diverse datasets that can capture the complexity of real-world engine deterioration.

Method: Multi-stage framework with stochastic exploration (emulating human trial-and-error) followed by reasoning-guided phase (goal-driven interactions), supporting multi-stride task generation across applications.

Result: Trained Qwen2.5-VL-7B on GUI-ReWalk dataset, achieving superior coverage of interaction flows, higher trajectory entropy, and more realistic user intent across multiple benchmarks including Screenspot-Pro, OSWorld-G, UI-Vision, AndroidControl, and GUI-Odyssey.

Conclusion: GUI-ReWalk provides a scalable and data-efficient framework for advancing GUI agent research and enabling robust real-world automation by producing data that better reflects human-computer interaction.

Abstract: Graphical User Interface (GUI) Agents, powered by large language and vision-language models, hold promise for enabling end-to-end automation in digital environments. However, their progress is fundamentally constrained by the scarcity of scalable, high-quality trajectory data. Existing data collection strategies either rely on costly and inconsistent manual annotations or on synthetic generation methods that trade off between diversity and meaningful task coverage. To bridge this gap, we present GUI-ReWalk: a reasoning-enhanced, multi-stage framework for synthesizing realistic and diverse GUI trajectories. GUI-ReWalk begins with a stochastic exploration phase that emulates human trial-and-error behaviors, and progressively transitions into a reasoning-guided phase where inferred goals drive coherent and purposeful interactions. Moreover, it supports multi-stride task generation, enabling the construction of long-horizon workflows across multiple applications. By combining randomness for diversity with goal-aware reasoning for structure, GUI-ReWalk produces data that better reflects the intent-aware, adaptive nature of human-computer interaction. We further train Qwen2.5-VL-7B on the GUI-ReWalk dataset and evaluate it across multiple benchmarks, including Screenspot-Pro, OSWorld-G, UI-Vision, AndroidControl, and GUI-Odyssey. Results demonstrate that GUI-ReWalk enables superior coverage of diverse interaction flows, higher trajectory entropy, and more realistic user intent. These findings establish GUI-ReWalk as a scalable and data-efficient framework for advancing GUI agent research and enabling robust real-world automation.

Method: Modified FSNet for single-pass incremental learning using pseudo targets generated by linear regression on input sequences. The model updates continuously using forecast errors and benefits from associative memory and adaptive structure mechanisms.

Result: Achieved 0.00197 RMSE and 0.00154 MAE on smooth degradation datasets, and 0.01588 RMSE and 0.01234 MAE on datasets with irregular degradation and capacity regeneration spikes.

Conclusion: iFSNet successfully enables online incremental multistep battery prognosis with immediate model adaptation, outperforming traditional offline methods that require retraining cycles.

Abstract: Data-driven models accurately perform early battery prognosis to prevent equipment failure and further safety hazards. Most existing machine learning (ML) models work in offline mode which must consider their retraining post-deployment every time new data distribution is encountered. Hence, there is a need for an online ML approach where the model can adapt to varying distributions. However, existing online incremental multistep forecasts are a great challenge as there is no way to correct the model of its forecasts at the current instance. Also, these methods need to wait for a considerable amount of time to acquire enough streaming data before retraining. In this study, we propose iFSNet (incremental Fast and Slow learning Network) which is a modified version of FSNet for a single-pass mode (sample-by-sample) to achieve multistep forecasting using pseudo targets. It uses a simple linear regressor of the input sequence to extrapolate pseudo future samples (pseudo targets) and calculate the loss from the rest of the forecast and keep updating the model. The model benefits from the associative memory and adaptive structure mechanisms of FSNet, at the same time the model incrementally improves by using pseudo targets. The proposed model achieved 0.00197 RMSE and 0.00154 MAE on datasets with smooth degradation trajectories while it achieved 0.01588 RMSE and 0.01234 MAE on datasets having irregular degradation trajectories with capacity regeneration spikes.

Method: Formulate an optimization program for optimal steering by finding the nearest ideal distribution in KL-divergence, with efficient algorithms for parametric families (e.g., normal, log-normal). Apply affine steering to LLM representations for bias reduction.

Result: Empirical results on synthetic and real-world datasets show improved fairness without diminishing utility, sometimes even improving utility. Demonstrated on multi-class classification tasks like occupation prediction from biographies.

Conclusion: Optimal steering techniques effectively reduce bias in ML models and LLMs, ensuring fair outcomes across different groups without compromising utility.

Abstract: To fix the ‘bias in, bias out’ problem in fair machine learning, it is important to steer feature distributions of data or internal representations of Large Language Models (LLMs) to ideal ones that guarantee group-fair outcomes. Previous work on fair generative models and representation steering could greatly benefit from provable fairness guarantees on the model output. We define a distribution as ideal if the minimizer of any cost-sensitive risk on it is guaranteed to have exact group-fair outcomes (e.g., demographic parity, equal opportunity)-in other words, it has no fairness-utility trade-off. We formulate an optimization program for optimal steering by finding the nearest ideal distribution in KL-divergence, and provide efficient algorithms for it when the underlying distributions come from well-known parametric families (e.g., normal, log-normal). Empirically, our optimal steering techniques on both synthetic and real-world datasets improve fairness without diminishing utility (and sometimes even improve utility). We demonstrate affine steering of LLM representations to reduce bias in multi-class classification, e.g., occupation prediction from a short biography in Bios dataset (De-Arteaga et al.). Furthermore, we steer internal representations of LLMs towards desired outputs so that it works equally well across different groups.

Method: Uses deep reinforcement learning with a heterogeneous graph neural network to represent policies, capturing diverse relationships among machines and processing steps for proactive decision-making. Includes scalability measures for machine-level actions.

Result: Evaluation on Intel’s Minifab model and SMT2020 testbed shows the trained policy increases throughput and decreases cycle time by about 1.8% each in the largest tested scenario.

Conclusion: The neural network-based approach demonstrates effectiveness in semiconductor manufacturing capacity planning, outperforming traditional heuristic methods by better handling complex process interactions.

Abstract: In manufacturing, capacity planning is the process of allocating production resources in accordance with variable demand. The current industry practice in semiconductor manufacturing typically applies heuristic rules to prioritize actions, such as future change lists that account for incoming machine and recipe dedications. However, while offering interpretability, heuristics cannot easily account for the complex interactions along the process flow that can gradually lead to the formation of bottlenecks. Here, we present a neural network-based model for capacity planning on the level of individual machines, trained using deep reinforcement learning. By representing the policy using a heterogeneous graph neural network, the model directly captures the diverse relationships among machines and processing steps, allowing for proactive decision-making. We describe several measures taken to achieve sufficient scalability to tackle the vast space of possible machine-level actions. Our evaluation results cover Intel’s small-scale Minifab model and preliminary experiments using the popular SMT2020 testbed. In the largest tested scenario, our trained policy increases throughput and decreases cycle time by about 1.8% each.

Method: The authors leverage on-average model stability to derive generalization bounds for both convex and strongly convex problems, without requiring the restrictive Lipschitzness assumption that limited previous analyses.

Result: The analysis demonstrates a linear speedup with respect to batch size in the convex setting, providing improved generalization bounds that better capture the optimization-generalization relationship.

Conclusion: This work fills an important gap in understanding Lookahead’s generalization capabilities by providing rigorous stability-based analysis that removes restrictive assumptions and reveals key relationships between optimization parameters and generalization performance.

Abstract: The Lookahead optimizer enhances deep learning models by employing a dual-weight update mechanism, which has been shown to improve the performance of underlying optimizers such as SGD. However, most theoretical studies focus on its convergence on training data, leaving its generalization capabilities less understood. Existing generalization analyses are often limited by restrictive assumptions, such as requiring the loss function to be globally Lipschitz continuous, and their bounds do not fully capture the relationship between optimization and generalization. In this paper, we address these issues by conducting a rigorous stability and generalization analysis of the Lookahead optimizer with minibatch SGD. We leverage on-average model stability to derive generalization bounds for both convex and strongly convex problems without the restrictive Lipschitzness assumption. Our analysis demonstrates a linear speedup with respect to the batch size in the convex setting.

Method: MCTD-ME integrates masked diffusion models with tree search, using biophysical-fidelity-enhanced diffusion denoising as rollout engine. It employs multiple experts of varying capacities guided by pLDDT-based masking schedule, and introduces PH-UCT-ME selection rule for expert ensembles.

Result: On inverse folding tasks (CAMEO and PDB benchmarks), MCTD-ME outperforms single-expert and unguided baselines in both sequence recovery (AAR) and structural similarity (scTM), with gains increasing for longer proteins and benefiting from multi-expert guidance.

Conclusion: The framework is model-agnostic and applicable beyond inverse folding, including de novo protein engineering and multi-objective molecular generation, demonstrating the effectiveness of combining diffusion models with tree search for complex protein design problems.

Abstract: The goal of protein design is to generate amino acid sequences that fold into functional structures with desired properties. Prior methods combining autoregressive language models with Monte Carlo Tree Search (MCTS) struggle with long-range dependencies and suffer from an impractically large search space. We propose MCTD-ME, Monte Carlo Tree Diffusion with Multiple Experts, which integrates masked diffusion models with tree search to enable multi-token planning and efficient exploration. Unlike autoregressive planners, MCTD-ME uses biophysical-fidelity-enhanced diffusion denoising as the rollout engine, jointly revising multiple positions and scaling to large sequence spaces. It further leverages experts of varying capacities to enrich exploration, guided by a pLDDT-based masking schedule that targets low-confidence regions while preserving reliable residues. We propose a novel multi-expert selection rule (PH-UCT-ME) extends predictive-entropy UCT to expert ensembles. On the inverse folding task (CAMEO and PDB benchmarks), MCTD-ME outperforms single-expert and unguided baselines in both sequence recovery (AAR) and structural similarity (scTM), with gains increasing for longer proteins and benefiting from multi-expert guidance. More generally, the framework is model-agnostic and applicable beyond inverse folding, including de novo protein engineering and multi-objective molecular generation.

Method: SABER connects two intermediate layers (s < e) through a residual connection to bypass safety mechanisms, leveraging the finding that safety features are concentrated in middle-to-late layers.

Result: Achieves 51% improvement over best-performing baseline on HarmBench test set with only marginal perplexity shift on validation set.

Conclusion: The method demonstrates that LLM safety mechanisms can be effectively bypassed through targeted layer manipulation, highlighting vulnerabilities in current alignment approaches.

Abstract: Large Language Models (LLMs) with safe-alignment training are powerful instruments with robust language comprehension capabilities. These models typically undergo meticulous alignment procedures involving human feedback to ensure the acceptance of safe inputs while rejecting harmful or unsafe ones. However, despite their massive scale and alignment efforts, LLMs remain vulnerable to jailbreak attacks, where malicious users manipulate the model to produce harmful outputs that it was explicitly trained to avoid. In this study, we find that the safety mechanisms in LLMs are predominantly embedded in the middle-to-late layers. Building on this insight, we introduce a novel white-box jailbreak method, SABER (Safety Alignment Bypass via Extra Residuals), which connects two intermediate layers $s$ and $e$ such that $s < e$, through a residual connection. Our approach achieves a 51% improvement over the best-performing baseline on the HarmBench test set. Furthermore, SABER induces only a marginal shift in perplexity when evaluated on the HarmBench validation set. The source code is publicly available at https://github.com/PalGitts/SABER.

Method: LSRE trains an autoencoder to map high-dimensional problem features to a 2D latent space, performs uniform-grid sampling for diverse hidden representations, and uses genetic programming to reverse engineer function formulas with minimal L2-distance to create Diverse-BBO problem set.

Result: MetaBBOs trained on Diverse-BBO show superior generalization performance on both synthetic and realistic scenarios compared to existing training sets. Ablation studies confirm LSRE’s effectiveness.

Conclusion: LSRE successfully generates diverse training instances that improve MetaBBO generalization, with design choices validated through ablation studies revealing insights on instance diversity and generalization relationships.

Abstract: To relieve intensive human-expertise required to design optimization algorithms, recent Meta-Black-Box Optimization (MetaBBO) researches leverage generalization strength of meta-learning to train neural network-based algorithm design policies over a predefined training problem set, which automates the adaptability of the low-level optimizers on unseen problem instances. Currently, a common training problem set choice in existing MetaBBOs is well-known benchmark suites CoCo-BBOB. Although such choice facilitates the MetaBBO’s development, problem instances in CoCo-BBOB are more or less limited in diversity, raising the risk of overfitting of MetaBBOs, which might further results in poor generalization. In this paper, we propose an instance generation approach, termed as \textbf{LSRE}, which could generate diverse training problem instances for MetaBBOs to learn more generalizable policies. LSRE first trains an autoencoder which maps high-dimensional problem features into a 2-dimensional latent space. Uniform-grid sampling in this latent space leads to hidden representations of problem instances with sufficient diversity. By leveraging a genetic-programming approach to search function formulas with minimal L2-distance to these hidden representations, LSRE reverse engineers a diversified problem set, termed as \textbf{Diverse-BBO}. We validate the effectiveness of LSRE by training various MetaBBOs on Diverse-BBO and observe their generalization performances on either synthetic or realistic scenarios. Extensive experimental results underscore the superiority of Diverse-BBO to existing training set choices in MetaBBOs. Further ablation studies not only demonstrate the effectiveness of design choices in LSRE, but also reveal interesting insights on instance diversity and MetaBBO’s generalization.

Method: ThermalGuardian generates test models using mutation rules for temperature-sensitive operators, simulates GPU temperature fluctuations using Newton’s law of cooling, and controls GPU frequency based on real-time temperature.

Result: The method identifies critical quality issues including delays/errors in compute-intensive operators, precision errors in high/mixed-precision operators, and synchronization issues in time-series operators.

Conclusion: ThermalGuardian is the first testing approach that considers temperature effects on automotive deep learning frameworks, bridging a critical gap in ensuring reliable autonomous driving systems.

Abstract: Deep learning models play a vital role in autonomous driving systems, supporting critical functions such as environmental perception. To accelerate model inference, these deep learning models’ deployment relies on automotive deep learning frameworks, for example, PaddleInference in Apollo and TensorRT in AutoWare. However, unlike deploying deep learning models on the cloud, vehicular environments experience extreme ambient temperatures varying from -40{\deg}C to 50{\deg}C, significantly impacting GPU temperature. Additionally, heats generated when computing further lead to the GPU temperature increase. These temperature fluctuations lead to dynamic GPU frequency adjustments through mechanisms such as DVFS. However, automotive deep learning frameworks are designed without considering the impact of temperature-induced frequency variations. When deployed on temperature-varying GPUs, these frameworks suffer critical quality issues: compute-intensive operators face delays or errors, high/mixed-precision operators suffer from precision errors, and time-series operators suffer from synchronization issues. The above quality issues cannot be detected by existing deep learning framework testing methods because they ignore temperature’s effect on the deep learning framework quality. To bridge this gap, we propose ThermalGuardian, the first automotive deep learning framework testing method under temperature-varying environments. Specifically, ThermalGuardian generates test input models using model mutation rules targeting temperature-sensitive operators, simulates GPU temperature fluctuations based on Newton’s law of cooling, and controls GPU frequency based on real-time GPU temperature.

Method: Analyzed various ML failures (reversal curse, agent navigation) and showed how an oracle retrieval mechanism can improve generalization. Explored essential components for effective retrieval, including within-example in-context learning.

Result: Systems with retrieval mechanisms can use learning experiences more flexibly and generalize better across challenges. Within-example in-context learning is crucial for using information across retrieved examples.

Conclusion: Lack of latent learning contributes to ML’s data inefficiency compared to natural intelligence. Retrieval methods complement parametric learning to improve generalization, with episodic memory being a key solution component.

Abstract: When do machine learning systems fail to generalize, and what mechanisms could improve their generalization? Here, we draw inspiration from cognitive science to argue that one weakness of machine learning systems is their failure to exhibit latent learning – learning information that is not relevant to the task at hand, but that might be useful in a future task. We show how this perspective links failures ranging from the reversal curse in language modeling to new findings on agent-based navigation. We then highlight how cognitive science points to episodic memory as a potential part of the solution to these issues. Correspondingly, we show that a system with an oracle retrieval mechanism can use learning experiences more flexibly to generalize better across many of these challenges. We also identify some of the essential components for effectively using retrieval, including the importance of within-example in-context learning for acquiring the ability to use information across retrieved examples. In summary, our results illustrate one possible contributor to the relative data inefficiency of current machine learning systems compared to natural intelligence, and help to understand how retrieval methods can complement parametric learning to improve generalization.

Method: The authors construct Muon-VR2 by incorporating a variance-reduction mechanism into the Muon framework. They provide rigorous convergence analysis under stochastic non-convex settings and Polyak-Łojasiewicz condition.

Result: Muon-VR2 achieves optimal Õ(T^{-1/3}) convergence rate, matching the theoretical lower bound. Experiments on CIFAR-10 and C4 benchmarks confirm the theoretical findings on per-iteration convergence.

Conclusion: This work provides the first proof of optimality for a Muon-style optimizer and clarifies the path toward developing more practically efficient, accelerated variants.

Abstract: The Muon optimizer has demonstrated remarkable empirical success in handling matrix-structured parameters for training neural networks. However, a significant gap persists between its practical performance and theoretical understanding. Existing analyses indicate that the standard Muon variant achieves only a suboptimal convergence rate of $\mathcal{O}(T^{-1/4})$ in stochastic non-convex settings, where $T$ denotes the number of iterations. To explore the theoretical limits of the Muon framework, we construct and analyze a variance-reduced variant, termed Muon-VR2. We provide the first rigorous proof that incorporating a variance-reduction mechanism enables Muon-VR2 to attain an optimal convergence rate of $\tilde{\mathcal{O}}(T^{-1/3})$, thereby matching the theoretical lower bound for this class of problems. Moreover, our analysis establishes convergence guarantees for Muon variants under the Polyak-{\L}ojasiewicz (P{\L}) condition. Extensive experiments on vision (CIFAR-10) and language (C4) benchmarks corroborate our theoretical findings on per-iteration convergence. Overall, this work provides the first proof of optimality for a Muon-style optimizer and clarifies the path toward developing more practically efficient, accelerated variants.

Method: Proposes EvoBrain with two-stream Mamba architecture integrated with GCN enhanced by Laplacian Positional Encoding, featuring explicitly dynamic graph structures where both nodes and edges evolve over time, following neurological insights.

Result: Achieves 23% improvement in AUROC and 30% improvement in F1 score compared to dynamic GNN baseline, with broad evaluations on challenging early seizure prediction tasks.

Conclusion: Theoretical analysis demonstrates expressivity advantage of explicit dynamic modeling and time-then-graph approach, with EvoBrain providing a novel and efficient solution for improved seizure detection performance.

Abstract: Dynamic GNNs, which integrate temporal and spatial features in Electroencephalography (EEG) data, have shown great potential in automating seizure detection. However, fully capturing the underlying dynamics necessary to represent brain states, such as seizure and non-seizure, remains a non-trivial task and presents two fundamental challenges. First, most existing dynamic GNN methods are built on temporally fixed static graphs, which fail to reflect the evolving nature of brain connectivity during seizure progression. Second, current efforts to jointly model temporal signals and graph structures and, more importantly, their interactions remain nascent, often resulting in inconsistent performance. To address these challenges, we present the first theoretical analysis of these two problems, demonstrating the effectiveness and necessity of explicit dynamic modeling and time-then-graph dynamic GNN method. Building on these insights, we propose EvoBrain, a novel seizure detection model that integrates a two-stream Mamba architecture with a GCN enhanced by Laplacian Positional Encoding, following neurological insights. Moreover, EvoBrain incorporates explicitly dynamic graph structures, allowing both nodes and edges to evolve over time. Our contributions include (a) a theoretical analysis proving the expressivity advantage of explicit dynamic modeling and time-then-graph over other approaches, (b) a novel and efficient model that significantly improves AUROC by 23% and F1 score by 30%, compared with the dynamic GNN baseline, and (c) broad evaluations of our method on the challenging early seizure prediction tasks.

Method: Uses graph neural networks to exploit inter- and intra-modal correlations between satellite images and meteorological station time series data, enabling probabilistic forecasts with 15-minute resolution up to 24 hours ahead.

Result: Achieves a normalized mean absolute error of 6.1% over the forecasting horizon on a dataset of 127 locations across Switzerland, competitive with commercial numerical weather prediction services.

Conclusion: SolarCrossFormer provides robust, high-resolution forecasting that can incorporate new data without retraining and produce forecasts for locations without input data, making it suitable for real-life grid operations.

Abstract: Accurate day-ahead forecasts of solar irradiance are required for the large-scale integration of solar photovoltaic (PV) systems into the power grid. However, current forecasting solutions lack the temporal and spatial resolution required by system operators. In this paper, we introduce SolarCrossFormer, a novel deep learning model for day-ahead irradiance forecasting, that combines satellite images and time series from a ground-based network of meteorological stations. SolarCrossFormer uses novel graph neural networks to exploit the inter- and intra-modal correlations of the input data and improve the accuracy and resolution of the forecasts. It generates probabilistic forecasts for any location in Switzerland with a 15-minute resolution for horizons up to 24 hours ahead. One of the key advantages of SolarCrossFormer its robustness in real life operations. It can incorporate new time-series data without retraining the model and, additionally, it can produce forecasts for locations without input data by using only their coordinates. Experimental results over a dataset of one year and 127 locations across Switzerland show that SolarCrossFormer yield a normalized mean absolute error of 6.1 % over the forecasting horizon. The results are competitive with those achieved by a commercial numerical weather prediction service.

Method: HyP-ASO combines a customized formula that uses feasible solutions to calculate variable selection probabilities with an RL policy network that predicts neighborhood size for adaptive search optimization.

Result: Extensive experiments show HyP-ASO significantly outperforms existing LNS-based approaches for large-scale ILPs and demonstrates lightweight, highly scalable performance.

Conclusion: The proposed framework is well-suited for solving large-scale ILPs due to its superior performance and scalability compared to existing methods.

Abstract: Directly solving large-scale Integer Linear Programs (ILPs) using traditional solvers is slow due to their NP-hard nature. While recent frameworks based on Large Neighborhood Search (LNS) can accelerate the solving process, their performance is often constrained by the difficulty in generating sufficiently effective neighborhoods. To address this challenge, we propose HyP-ASO, a hybrid policy-based adaptive search optimization framework that combines a customized formula with deep Reinforcement Learning (RL). The formula leverages feasible solutions to calculate the selection probabilities for each variable in the neighborhood generation process, and the RL policy network predicts the neighborhood size. Extensive experiments demonstrate that HyP-ASO significantly outperforms existing LNS-based approaches for large-scale ILPs. Additional experiments show it is lightweight and highly scalable, making it well-suited for solving large-scale ILPs.

Method: Developed Tsururu as a flexible Python library that enables combinations of global and multivariate approaches, multi-step-ahead forecasting strategies, and seamless integration with various forecasting models.

Result: Tsururu is available as an open-source library at https://github.com/sb-ai-lab/tsururu, providing practical tools for time series forecasting that connect research advancements with industry needs.

Conclusion: Tsururu successfully addresses the gap in time series forecasting by providing a flexible framework that combines advanced research approaches with practical industry requirements through an accessible Python library.

Abstract: While current time series research focuses on developing new models, crucial questions of selecting an optimal approach for training such models are underexplored. Tsururu, a Python library introduced in this paper, bridges SoTA research and industry by enabling flexible combinations of global and multivariate approaches and multi-step-ahead forecasting strategies. It also enables seamless integration with various forecasting models. Available at https://github.com/sb-ai-lab/tsururu .

Method: Developed Heat Kernel Distance (HKD) transformation with convergence guarantees. Created two algorithms: HK-MVFC for central analysis and FedHK-MVFC for federated learning using differential privacy and secure aggregation.

Result: 8-12% increase in clustering accuracy, 70% reduced communication, and 98.2% efficiency retention over centralized methods. Validated on 10,000 patient records across two hospitals with ECG, cardiac imaging, and behavioral data.

Conclusion: Establishes a new standard for geometry-aware federated learning in healthcare, transforming advanced mathematics into practical solutions for analyzing sensitive medical data with both theoretical rigor and clinical relevance.

Abstract: In the realm of distributed AI and privacy-focused medical applications, we propose a framework for multi-view clustering that links quantum field theory with federated healthcare analytics. Our method uses heat-kernel coefficients from spectral analysis to convert Euclidean distances into geometry-aware similarity measures, capturing the structure of diverse medical data. We lay this out through the Heat Kernel Distance (HKD) transformation with convergence guarantees. Two algorithms are developed: Heat Kernel-Enhanced Multi-View Fuzzy Clustering (HK-MVFC) for central analysis, and Federated Heat Kernel Multi-View Fuzzy Clustering (FedHK-MVFC) for secure, privacy-preserving learning across hospitals using differential privacy and secure aggregation to facilitate HIPAA-compliant collaboration. Tests on synthetic datasets of cardiovascular patients show an $8-12 %$ increase in clustering accuracy, $70 %$ reduced communication, and $98.2 %$ efficiency retention over centralized methods. Validated on 10,000 patient records across two hospitals, it proves useful for collaborative phenotyping involving ECG, cardiac imaging, and behavioral data. Our theoretical contributions include update rules with proven convergence, adaptive view weighting, and privacy-preserving protocols. This presents a new standard for geometry-aware federated learning in healthcare, turning advanced math into workable solutions for analyzing sensitive medical data while ensuring both rigor and clinical relevance.

Method: Leverage the rich semantic latent space of Vision Foundation Models to generate synthetic data, then train a simple linear classifier using a mixture of real and synthetic data for long-tail classification.

Result: Sets new state-of-the-art for CIFAR-100-LT benchmark and demonstrates strong performance on Places-LT benchmark, with significant computational efficiency gains by reducing trainable parameters to just those in the linear model.

Conclusion: The proposed simple and effective approach is highly efficient and adaptable, successfully addressing imbalanced classification challenges while maintaining strong performance with minimal computational requirements.

Abstract: Imbalanced classification datasets pose significant challenges in machine learning, often leading to biased models that perform poorly on underrepresented classes. With the rise of foundation models, recent research has focused on the full, partial, and parameter-efficient fine-tuning of these models to deal with long-tail classification. Despite the impressive performance of these works on the benchmark datasets, they still fail to close the gap with the networks trained using the balanced datasets and still require substantial computational resources, even for relatively smaller datasets. Underscoring the importance of computational efficiency and simplicity, in this work we propose a novel framework that leverages the rich semantic latent space of Vision Foundation Models to generate synthetic data and train a simple linear classifier using a mixture of real and synthetic data for long-tail classification. The computational efficiency gain arises from the number of trainable parameters that are reduced to just the number of parameters in the linear model. Our method sets a new state-of-the-art for the CIFAR-100-LT benchmark and demonstrates strong performance on the Places-LT benchmark, highlighting the effectiveness and adaptability of our simple and effective approach.

Method: The authors created a large public dataset with 246 pediatric patients, over 40,000 annotated cells, and developed AI methods for cell detection, cell classification (33 classes), and diagnosis prediction using predicted cell counts.

Result: The AI models achieved: 0.96 average precision for cell detection, 0.98 AUC and 0.61 F1-score for cell classification, and 0.90 mean F1-score for diagnosis prediction.

Conclusion: The proposed approaches demonstrate usefulness for AI-assisted diagnostics, and the public dataset will foster further research to improve leukemia diagnosis precision and patient outcomes.

Abstract: Leukemia diagnosis primarily relies on manual microscopic analysis of bone marrow morphology supported by additional laboratory parameters, making it complex and time consuming. While artificial intelligence (AI) solutions have been proposed, most utilize private datasets and only cover parts of the diagnostic pipeline. Therefore, we present a large, high-quality, publicly available leukemia bone marrow dataset spanning the entire diagnostic process, from cell detection to diagnosis. Using this dataset, we further propose methods for cell detection, cell classification, and diagnosis prediction. The dataset comprises 246 pediatric patients with diagnostic, clinical and laboratory information, over 40 000 cells with bounding box annotations and more than 28 000 of these with high-quality class labels, making it the most comprehensive dataset publicly available. Evaluation of the AI models yielded an average precision of 0.96 for the cell detection, an area under the curve of 0.98, and an F1-score of 0.61 for the 33-class cell classification, and a mean F1-score of 0.90 for the diagnosis prediction using predicted cell counts. While the proposed approaches demonstrate their usefulness for AI-assisted diagnostics, the dataset will foster further research and development in the field, ultimately contributing to more precise diagnoses and improved patient outcomes.

Method: Proposes a learning-to-unlearn framework that incorporates transformations during the learning process to reduce memorization of specific instances. Uses transformation composition to provably bound instance-specific information.

Result: ToFU outperforms existing FU baselines on CIFAR-10, CIFAR-100, and MUFAC benchmark, enhances performance when integrated with current methods, and reduces unlearning time.

Conclusion: ToFU provides an effective plug-and-play framework that shifts from post-hoc unlearning to proactive memorization reduction during learning, making federated unlearning more efficient and practical.

Abstract: Neural networks unintentionally memorize training data, creating privacy risks in federated learning (FL) systems, such as inference and reconstruction attacks on sensitive data. To mitigate these risks and to comply with privacy regulations, Federated Unlearning (FU) has been introduced to enable participants in FL systems to remove their data’s influence from the global model. However, current FU methods primarily act post-hoc, struggling to efficiently erase information deeply memorized by neural networks. We argue that effective unlearning necessitates a paradigm shift: designing FL systems inherently amenable to forgetting. To this end, we propose a learning-to-unlearn Transformation-guided Federated Unlearning (ToFU) framework that incorporates transformations during the learning process to reduce memorization of specific instances. Our theoretical analysis reveals how transformation composition provably bounds instance-specific information, directly simplifying subsequent unlearning. Crucially, ToFU can work as a plug-and-play framework that improves the performance of existing FU methods. Experiments on CIFAR-10, CIFAR-100, and the MUFAC benchmark show that ToFU outperforms existing FU baselines, enhances performance when integrated with current methods, and reduces unlearning time.

Method: SAGE integrates a shared sampling scheme for efficiency and a tailored training strategy for quality preservation. It reduces total sampling steps by sharing early-stage sampling across semantically similar queries.

Result: Extensive experiments show SAGE reduces sampling cost by 25.5% while improving generation quality: 5.0% lower FID, 5.4% higher CLIP score, and 160% higher diversity over baselines.

Conclusion: SAGE demonstrates that sharing early-stage sampling across semantically similar queries can significantly reduce diffusion model sampling costs while maintaining or even improving generation quality.

Abstract: Diffusion models manifest evident benefits across diverse domains, yet their high sampling cost, requiring dozens of sequential model evaluations, remains a major limitation. Prior efforts mainly accelerate sampling via optimized solvers or distillation, which treat each query independently. In contrast, we reduce total number of steps by sharing early-stage sampling across semantically similar queries. To enable such efficiency gains without sacrificing quality, we propose SAGE, a semantic-aware shared sampling framework that integrates a shared sampling scheme for efficiency and a tailored training strategy for quality preservation. Extensive experiments show that SAGE reduces sampling cost by 25.5%, while improving generation quality with 5.0% lower FID, 5.4% higher CLIP, and 160% higher diversity over baselines.

Method: Empirical evaluation of FWMs (using FM prior knowledge for simulation) and FAs (using FM reasoning for decision-making) in grid-world environments suitable for current LLMs.

Result: LLM improvements translate to better FWMs and FAs; FAs can provide excellent policies for simple environments; FWMs coupled with RL agents show promise for complex settings with partial observability and stochastic elements.

Conclusion: Both FWM and FA approaches are promising for improving RL sample efficiency, with FAs working well for simple environments and FWMs showing particular potential for complex, partially observable settings.

Abstract: While reinforcement learning from scratch has shown impressive results in solving sequential decision-making tasks with efficient simulators, real-world applications with expensive interactions require more sample-efficient agents. Foundation models (FMs) are natural candidates to improve sample efficiency as they possess broad knowledge and reasoning capabilities, but it is yet unclear how to effectively integrate them into the reinforcement learning framework. In this paper, we anticipate and, most importantly, evaluate two promising strategies. First, we consider the use of foundation world models (FWMs) that exploit the prior knowledge of FMs to enable training and evaluating agents with simulated interactions. Second, we consider the use of foundation agents (FAs) that exploit the reasoning capabilities of FMs for decision-making. We evaluate both approaches empirically in a family of grid-world environments that are suitable for the current generation of large language models (LLMs). Our results suggest that improvements in LLMs already translate into better FWMs and FAs; that FAs based on current LLMs can already provide excellent policies for sufficiently simple environments; and that the coupling of FWMs and reinforcement learning agents is highly promising for more complex settings with partial observability and stochastic elements.

Method: Proposes AIGB-Pearl which constructs a non-bootstrapped trajectory evaluator to assign rewards and guide policy search, with three key techniques: LLM-based architecture, hybrid point-wise and pair-wise losses, and adaptive integration of expert feedback.

Result: Extensive experiments on both simulated and real-world advertising systems demonstrate state-of-the-art performance.

Conclusion: AIGB-Pearl effectively addresses the limitations of existing AIGB methods and achieves superior performance in auto-bidding applications.

Abstract: Auto-bidding is an essential tool for advertisers to enhance their advertising performance. Recent progress has shown that AI-Generated Bidding (AIGB), which formulates the auto-bidding as a trajectory generation task and trains a conditional diffusion-based planner on offline data, achieves superior and stable performance compared to typical offline reinforcement learning (RL)-based auto-bidding methods. However, existing AIGB methods still encounter a performance bottleneck due to their neglect of fine-grained generation quality evaluation and inability to explore beyond static datasets. To address this, we propose AIGB-Pearl (\emph{Planning with EvAluator via RL}), a novel method that integrates generative planning and policy optimization. The key to AIGB-Pearl is to construct a non-bootstrapped \emph{trajectory evaluator} to assign rewards and guide policy search, enabling the planner to optimize its generation quality iteratively through interaction. Furthermore, to enhance trajectory evaluator accuracy in offline settings, we incorporate three key techniques: (i) a Large Language Model (LLM)-based architecture for better representational capacity, (ii) hybrid point-wise and pair-wise losses for better score learning, and (iii) adaptive integration of expert feedback for better generalization ability. Extensive experiments on both simulated and real-world advertising systems demonstrate the state-of-the-art performance of our approach.

Method: Model the alignment loop as a two-stage cascade U→H→Y given S, with cognitive capacity C_cog|S and average total capacity C_tot|S. Develop a capacity-coupled Alignment Performance Interval with a Fano lower bound and PAC-Bayes upper bound controlled by the same channel capacity.

Result: The analysis shows that with fixed value complexity and capacity, adding labels alone cannot cross performance bounds; lower risk on complex targets requires capacity growing with log M; and once capacity saturates, further optimization fits channel regularities (sycophancy/reward hacking).

Conclusion: Alignment should be viewed as interface engineering: measure and allocate limited capacity, manage task complexity, and strategically decide where information is spent to avoid systematic deviations.

Abstract: Large language models improve with scale, yet feedback-based alignment still exhibits systematic deviations from intended behavior. Motivated by bounded rationality in economics and cognitive science, we view judgment as resource-limited and feedback as a constrained channel. On this basis, we model the loop as a two-stage cascade $U \to H \to Y$ given $S$, with cognitive capacity $C_{\text{cog}|S}$ and average total capacity $\bar{C}{\text{tot}|S}$. Our main result is a capacity-coupled Alignment Performance Interval. It pairs a data size-independent Fano lower bound proved on a separable codebook mixture with a PAC-Bayes upper bound whose KL term is controlled by the same channel via $m , \bar{C}{\text{tot}|S}$. The PAC-Bayes bound becomes an upper bound on the same true risk when the canonical observable loss is used and the dataset is drawn from the same mixture. Under these matched conditions, both limits are governed by a single capacity. Consequences include that, with value complexity and capacity fixed, adding labels alone cannot cross the bound; attaining lower risk on more complex targets requires capacity that grows with $\log M$; and once useful signal saturates capacity, further optimization tends to fit channel regularities, consistent with reports of sycophancy and reward hacking. The analysis views alignment as interface engineering: measure and allocate limited capacity, manage task complexity, and decide where information is spent.

Method: Proposes two key innovations: (1) extreme bandit allocation strategy for identifying globally optimal expressions with finite-time performance guarantees, and (2) evolution-inspired state-jumping actions (mutation and crossover) that enable non-local transitions and reshape the reward landscape.

Result: The approach achieves competitive performance with state-of-the-art libraries in recovery rate and attains favorable positions on the Pareto frontier of accuracy versus model complexity across various datasets.

Conclusion: The improved MCTS framework effectively addresses limitations of traditional methods through innovative bandit strategies and evolutionary operations, demonstrating superior performance in symbolic regression tasks.

Abstract: Symbolic regression aims to discover concise, interpretable mathematical expressions that satisfy desired objectives, such as fitting data, posing a highly combinatorial optimization problem. While genetic programming has been the dominant approach, recent efforts have explored reinforcement learning methods for improving search efficiency. Monte Carlo Tree Search (MCTS), with its ability to balance exploration and exploitation through guided search, has emerged as a promising technique for symbolic expression discovery. However, its traditional bandit strategies and sequential symbol construction often limit performance. In this work, we propose an improved MCTS framework for symbolic regression that addresses these limitations through two key innovations: (1) an extreme bandit allocation strategy tailored for identifying globally optimal expressions, with finite-time performance guarantees under polynomial reward decay assumptions; and (2) evolution-inspired state-jumping actions such as mutation and crossover, which enable non-local transitions to promising regions of the search space. These state-jumping actions also reshape the reward landscape during the search process, improving both robustness and efficiency. We conduct a thorough numerical study to the impact of these improvements and benchmark our approach against existing symbolic regression methods on a variety of datasets, including both ground-truth and black-box datasets. Our approach achieves competitive performance with state-of-the-art libraries in terms of recovery rate, attains favorable positions on the Pareto frontier of accuracy versus model complexity. Code is available at https://github.com/PKU-CMEGroup/MCTS-4-SR.

Method: Embedding Bayesian Neural Networks into PINN architecture, using heat diffusion PDE as physical residual for transformer insulation degradation driven by thermal stress, with investigation of different prior distributions.

Result: B-PINN delivers more reliable prognostic predictions than dropout-PINN baseline by accurately quantifying predictive uncertainty, validated against finite element model with real measurements from a solar power plant.

Conclusion: The framework provides crucial capability for supporting robust and informed maintenance decision-making in critical power assets through principled uncertainty-aware predictions.

Abstract: Scientific Machine Learning (SciML) integrates physics and data into the learning process, offering improved generalization compared with purely data-driven models. Despite its potential, applications of SciML in prognostics remain limited, partly due to the complexity of incorporating partial differential equations (PDEs) for ageing physics and the scarcity of robust uncertainty quantification methods. This work introduces a Bayesian Physics-Informed Neural Network (B-PINN) framework for probabilistic prognostics estimation. By embedding Bayesian Neural Networks into the PINN architecture, the proposed approach produces principled, uncertainty-aware predictions. The method is applied to a transformer ageing case study, where insulation degradation is primarily driven by thermal stress. The heat diffusion PDE is used as the physical residual, and different prior distributions are investigated to examine their impact on predictive posterior distributions and their ability to encode a priori physical knowledge. The framework is validated against a finite element model developed and tested with real measurements from a solar power plant. Results, benchmarked against a dropout-PINN baseline, show that the proposed B-PINN delivers more reliable prognostic predictions by accurately quantifying predictive uncertainty. This capability is crucial for supporting robust and informed maintenance decision-making in critical power assets.

Method: Three-stage framework: 1) sampling and pre-ranking pose candidates, 2) iterative refinement, 3) post-ranking. Uses unified energy-based diffusion model with render-compare architecture for sim-to-real transfer.

Result: Outperforms conventional baselines (regression, matching, registration) and shows strong intra-category generalization to unseen CAD models.

Conclusion: The approach successfully integrates tactile pose estimation, tracking, and uncertainty estimation into a unified framework with robust real-world performance.

Abstract: Accurate estimation of the in-hand pose of an object based on its CAD model is crucial in both industrial applications and everyday tasks, ranging from positioning workpieces and assembling components to seamlessly inserting devices like USB connectors. While existing methods often rely on regression, feature matching, or registration techniques, achieving high precision and generalizability to unseen CAD models remains a significant challenge. In this paper, we propose a novel three-stage framework for in-hand pose estimation. The first stage involves sampling and pre-ranking pose candidates, followed by iterative refinement of these candidates in the second stage. In the final stage, post-ranking is applied to identify the most likely pose candidates. These stages are governed by a unified energy-based diffusion model, which is trained solely on simulated data. This energy model simultaneously generates gradients to refine pose estimates and produces an energy scalar that quantifies the quality of the pose estimates. Additionally, borrowing the idea from the computer vision domain, we incorporate a render-compare architecture within the energy-based score network to significantly enhance sim-to-real performance, as demonstrated by our ablation studies. We conduct comprehensive experiments to show that our method outperforms conventional baselines based on regression, matching, and registration techniques, while also exhibiting strong intra-category generalization to previously unseen CAD models. Moreover, our approach integrates tactile object pose estimation, pose tracking, and uncertainty estimation into a unified framework, enabling robust performance across a variety of real-world conditions.

Method: Applied influence analysis to DeepRx (fully convolutional receiver) using loss-relative influence with capacity-like binary cross-entropy loss, with first-order updates on beneficial samples and proposed second-order influence-aligned update strategy.

Result: Targeted fine-tuning using influence functions consistently improved bit error rate toward genie-aided performance, outperforming random fine-tuning in single-target scenarios, though multi-target adaptation was less effective.

Conclusion: Influence functions serve as both an interpretability tool and basis for efficient receiver adaptation, establishing their utility in wireless communications deep learning applications.

Abstract: We present the first use of influence functions for deep learning-based wireless receivers. Applied to DeepRx, a fully convolutional receiver, influence analysis reveals which training samples drive bit predictions, enabling targeted fine-tuning of poorly performing cases. We show that loss-relative influence with capacity-like binary cross-entropy loss and first-order updates on beneficial samples most consistently improves bit error rate toward genie-aided performance, outperforming random fine-tuning in single-target scenarios. Multi-target adaptation proved less effective, underscoring open challenges. Beyond experiments, we connect influence to self-influence corrections and propose a second-order, influence-aligned update strategy. Our results establish influence functions as both an interpretability tool and a basis for efficient receiver adaptation.

Method: Proposes AGF-TI with: 1) adversarial graph fusion scheme using min-max framework to learn robust consensus graph against distorted structures, 2) low-rank tensor learning to recover incomplete structures from high-order consistency, 3) anchor-based strategy for computational efficiency, and 4) efficient alternative optimization algorithm with theoretical convergence guarantee.

Result: Extensive experiments on various datasets show AGF-TI outperforms state-of-the-art methods in handling incomplete multi-view data.

Conclusion: AGF-TI effectively addresses the Sub-Cluster Problem in incomplete multi-view semi-supervised learning through adversarial graph fusion and tensor-based structure recovery, demonstrating superior performance over existing methods.

Abstract: View missing remains a significant challenge in graph-based multi-view semi-supervised learning, hindering their real-world applications. To address this issue, traditional methods introduce a missing indicator matrix and focus on mining partial structure among existing samples in each view for label propagation (LP). However, we argue that these disregarded missing samples sometimes induce discontinuous local structures, i.e., sub-clusters, breaking the fundamental smoothness assumption in LP. Consequently, such a Sub-Cluster Problem (SCP) would distort graph fusion and degrade classification performance. To alleviate SCP, we propose a novel incomplete multi-view semi-supervised learning method, termed AGF-TI. Firstly, we design an adversarial graph fusion scheme to learn a robust consensus graph against the distorted local structure through a min-max framework. By stacking all similarity matrices into a tensor, we further recover the incomplete structure from the high-order consistency information based on the low-rank tensor learning. Additionally, the anchor-based strategy is incorporated to reduce the computational complexity. An efficient alternative optimization algorithm combining a reduced gradient descent method is developed to solve the formulated objective, with theoretical convergence. Extensive experimental results on various datasets validate the superiority of our proposed AGF-TI as compared to state-of-the-art methods. Code is available at https://github.com/ZhangqiJiang07/AGF_TI.

Method: RLinf uses M2Flow paradigm to automatically break down high-level RL workflows at temporal and spatial dimensions, recomposing them into optimized execution flows. It employs context switching, elastic pipelining, adaptive communication, and profiling-guided scheduling.

Result: Extensive evaluations on reasoning RL and embodied RL tasks show RLinf consistently outperforms state-of-the-art systems with 1.1x-2.13x speedup in end-to-end training throughput.

Conclusion: RLinf demonstrates that addressing system flexibility through the M2Flow paradigm enables significant performance improvements in RL training, making it a promising solution for efficient RL system design.

Abstract: Reinforcement learning (RL) has demonstrated immense potential in advancing artificial general intelligence, agentic intelligence, and embodied intelligence. However, the inherent heterogeneity and dynamicity of RL workflows often lead to low hardware utilization and slow training on existing systems. In this paper, we present RLinf, a high-performance RL training system based on our key observation that the major roadblock to efficient RL training lies in system flexibility. To maximize flexibility and efficiency, RLinf is built atop a novel RL system design paradigm called macro-to-micro flow transformation (M2Flow), which automatically breaks down high-level, easy-to-compose RL workflows at both the temporal and spatial dimensions, and recomposes them into optimized execution flows. Supported by RLinf worker’s adaptive communication capability, we devise context switching and elastic pipelining to realize M2Flow transformation, and a profiling-guided scheduling policy to generate optimal execution plans. Extensive evaluations on both reasoning RL and embodied RL tasks demonstrate that RLinf consistently outperforms state-of-the-art systems, achieving 1.1x-2.13x speedup in end-to-end training throughput.

Method: Uses ensemble methods to model Q-value distributions for both demonstration and policy actions, with two uncertainty-aware approaches: probabilistic estimation of demonstration superiority likelihood, and advantage-based scaling by statistical significance.

Result: Achieves up to 14x performance gains in complex robotics tasks compared to existing approaches, while maintaining robustness to demonstration quality and quantity.

Conclusion: SPReD provides an effective framework for uncertainty-aware demonstration imitation that significantly outperforms existing methods through continuous regularization rather than binary decisions.

Abstract: In reinforcement learning with sparse rewards, demonstrations can accelerate learning, but determining when to imitate them remains challenging. We propose Smooth Policy Regularisation from Demonstrations (SPReD), a framework that addresses the fundamental question: when should an agent imitate a demonstration versus follow its own policy? SPReD uses ensemble methods to explicitly model Q-value distributions for both demonstration and policy actions, quantifying uncertainty for comparisons. We develop two complementary uncertainty-aware methods: a probabilistic approach estimating the likelihood of demonstration superiority, and an advantage-based approach scaling imitation by statistical significance. Unlike prevailing methods (e.g. Q-filter) that make binary imitation decisions, SPReD applies continuous, uncertainty-proportional regularisation weights, reducing gradient variance during training. Despite its computational simplicity, SPReD achieves remarkable gains in experiments across eight robotics tasks, outperforming existing approaches by up to a factor of 14 in complex tasks while maintaining robustness to demonstration quality and quantity. Our code is available at https://github.com/YujieZhu7/SPReD.

Method: Proposes Inverse Optimization Latent Variable Model (IO-LVM) that learns latent space of COP cost functions using estimated gradients of Fenchel-Young loss through non-differentiable deterministic solver, with solver in loop for reconstruction.

Result: Validated on ship/taxi routes and synthetic graphs, demonstrating ability to reconstruct paths/cycles, predict distributions, and yield interpretable latent representations.

Conclusion: IO-LVM effectively captures distributions over cost functions, enabling identification of diverse solution behaviors from different agents/conditions not available during training.

Abstract: Learning representations for solutions of constrained optimization problems (COPs) with unknown cost functions is challenging, as models like (Variational) Autoencoders struggle to enforce constraints when decoding structured outputs. We propose an Inverse Optimization Latent Variable Model (IO-LVM) that learns a latent space of COP cost functions from observed solutions and reconstructs feasible outputs by solving a COP with a solver in the loop. Our approach leverages estimated gradients of a Fenchel-Young loss through a non-differentiable deterministic solver to shape the latent space. Unlike standard Inverse Optimization or Inverse Reinforcement Learning methods, which typically recover a single or context-specific cost function, IO-LVM captures a distribution over cost functions, enabling the identification of diverse solution behaviors arising from different agents or conditions not available during the training process. We validate our method on real-world datasets of ship and taxi routes, as well as paths in synthetic graphs, demonstrating its ability to reconstruct paths and cycles, predict their distributions, and yield interpretable latent representations.

Method: The system collects online statements, encodes them using Universal Sentence Encoder into 512-dimensional vectors, trains an SVM classifier with 10-fold cross-validation, and uses tracking algorithms for temporal analysis of attitude changes.

Result: Achieved 81% accuracy in detecting extremism intentions and 97% accuracy for terrorism detection using 839 quotes, outperforming baseline n-gram features. Tracking algorithms successfully detected trends and sudden attitude changes.

Conclusion: The proposed ML-based system effectively identifies extremism and terrorism intentions from online statements and can track temporal changes in attitudes, demonstrating practical utility for security monitoring applications.

Abstract: This paper proposes an approach for automatically analysing and tracking statements in material gathered online and detecting whether the authors of the statements are likely to be involved in extremism or terrorism. The proposed system comprises: online collation of statements that are then encoded in a form amenable to machine learning (ML), an ML component to classify the encoded text, a tracker, and a visualisation system for analysis of results. The detection and tracking concept has been tested using quotes made by terrorists, extremists, campaigners, and politicians, obtained from wikiquote.org. A set of features was extracted for each quote using the state-of-the-art Universal Sentence Encoder (Cer et al. 2018), which produces 512-dimensional vectors. The data were used to train and test a support vector machine (SVM) classifier using 10-fold cross-validation. The system was able to correctly detect intentions and attitudes associated with extremism 81% of the time and terrorism 97% of the time, using a dataset of 839 quotes. This accuracy was higher than that which was achieved for a simple baseline system based on n-gram text features. Tracking techniques were also used to perform a temporal analysis of the data, with each quote considered to be a noisy measurement of a person’s state of mind. It was demonstrated that the tracking algorithms were able to detect both trends over time and sharp changes in attitude that could be attributed to major events.

Method: The authors adapt and extend the TSympNets architecture from previous work to handle non-autonomous Hamiltonian systems. They provide theoretical analysis including a universal approximation theorem for separable Hamiltonian systems and investigate approximation capabilities through numerical experiments.

Result: The paper proves that TSympNets can universally approximate separable Hamiltonian systems but cannot be extended to non-separable Hamiltonian systems. They also correct a mistake in a previous theorem about approximating symplectic maps.

Conclusion: TSympNets provide an effective framework for learning time-adaptive symplectic integrators for irregularly sampled data, with proven approximation capabilities for separable Hamiltonian systems, though with limitations for non-separable systems.

Abstract: Measurement data is often sampled irregularly i.e. not on equidistant time grids. This is also true for Hamiltonian systems. However, existing machine learning methods, which learn symplectic integrators, such as SympNets [20] and H'enonNets [4] still require training data generated by fixed step sizes. To learn time-adaptive symplectic integrators, an extension to SympNets, which we call TSympNets, was introduced in [20]. We adapt the architecture of TSympNets and extend them to non-autonomous Hamiltonian systems. So far the approximation qualities of TSympNets were unknown. We close this gap by providing a universal approximation theorem for separable Hamiltonian systems and show that it is not possible to extend it to non-separable Hamiltonian systems. To investigate these theoretical approximation capabilities, we perform different numerical experiments. Furthermore we fix a mistake in a proof of a substantial theorem [25, Theorem 2] for the approximation of symplectic maps in general, but specifically for symplectic machine learning methods.

Method: Nine algorithm-criterion combinations (3 regression methods × 3 selection criteria) are systematically paired and applied to isotropic and anisotropic hyperelasticity using synthetic and experimental datasets.

Result: All nine combinations performed consistently well, yielding highly accurate constitutive models for both isotropic and anisotropic materials, demonstrating viability beyond ℓ₁-based approaches.

Conclusion: The framework successfully broadens the range of viable discovery algorithms and provides systematic exploration of sparsity-performance-computation trade-offs in automated constitutive modeling.

Abstract: The automated discovery of constitutive models from data has recently emerged as a promising alternative to the traditional model calibration paradigm. In this work, we present a fully automated framework for constitutive model discovery that systematically pairs three sparse regression algorithms (Least Absolute Shrinkage and Selection Operator (LASSO), Least Angle Regression (LARS), and Orthogonal Matching Pursuit (OMP)) with three model selection criteria: $K$-fold cross-validation (CV), Akaike Information Criterion (AIC), and Bayesian Information Criterion (BIC). This pairing yields nine distinct algorithms for model discovery and enables a systematic exploration of the trade-off between sparsity, predictive performance, and computational cost. While LARS serves as an efficient path-based solver for the $\ell_1$-constrained problem, OMP is introduced as a tractable heuristic for $\ell_0$-regularized selection. The framework is applied to both isotropic and anisotropic hyperelasticity, utilizing both synthetic and experimental datasets. Results reveal that all nine algorithm-criterion combinations perform consistently well for the discovery of isotropic and anisotropic materials, yielding highly accurate constitutive models. These findings broaden the range of viable discovery algorithms beyond $\ell_1$-based approaches such as LASSO.

Method: Uses Forward Neural Networks and Transformer networks to capture complex spatiotemporal relationships between GNSS-derived features and wind dynamics from 5G signals.

Result: Achieves promising accuracy in wind retrieval and short-term forecasting (up to 30 minutes), showing superior agreement with observations compared to ERA5 reanalysis data, with robust performance across different forecast horizons and pressure levels.

Conclusion: Demonstrates the transformative potential of using non-traditional data sources and deep learning for cost-effective, scalable real-time atmospheric monitoring applications.

Abstract: Accurate atmospheric wind field information is crucial for various applications, including weather forecasting, aviation safety, and disaster risk reduction. However, obtaining high spatiotemporal resolution wind data remains challenging due to limitations in traditional in-situ observations and remote sensing techniques, as well as the computational expense and biases of numerical weather prediction (NWP) models. This paper introduces G-WindCast, a novel deep learning framework that leverages signal strength variations from 5G Global Navigation Satellite System (GNSS) signals to retrieve and forecast three-dimensional (3D) atmospheric wind fields. The framework utilizes Forward Neural Networks (FNN) and Transformer networks to capture complex, nonlinear, and spatiotemporal relationships between GNSS-derived features and wind dynamics. Our preliminary results demonstrate promising accuracy in both wind retrieval and short-term wind forecasting (up to 30 minutes lead time), with skill scores comparable to high-resolution NWP outputs in certain scenarios. The model exhibits robustness across different forecast horizons and pressure levels, and its predictions for wind speed and direction show superior agreement with observations compared to concurrent ERA5 reanalysis data. Furthermore, we show that the system can maintain excellent performance for localized forecasting even with a significantly reduced number of GNSS stations (e.g., around 100), highlighting its cost-effectiveness and scalability. This interdisciplinary approach underscores the transformative potential of exploiting non-traditional data sources and deep learning for advanced environmental monitoring and real-time atmospheric applications.

Method: Proposes Dual-Masked Autoencoder (DMAE) with two complementary pretext tasks: (1) reconstructing masked values based on visible attributes, and (2) estimating latent representations of masked features guided by a teacher encoder, plus feature-level alignment constraint.

Result: Comprehensive evaluations across classification, regression, and forecasting tasks demonstrate consistent and superior performance over competitive baselines.

Conclusion: DMAE learns temporally coherent and semantically rich representations through joint optimization of dual reconstruction objectives and feature alignment.

Abstract: Unsupervised multivariate time series (MTS) representation learning aims to extract compact and informative representations from raw sequences without relying on labels, enabling efficient transfer to diverse downstream tasks. In this paper, we propose Dual-Masked Autoencoder (DMAE), a novel masked time-series modeling framework for unsupervised MTS representation learning. DMAE formulates two complementary pretext tasks: (1) reconstructing masked values based on visible attributes, and (2) estimating latent representations of masked features, guided by a teacher encoder. To further improve representation quality, we introduce a feature-level alignment constraint that encourages the predicted latent representations to align with the teacher’s outputs. By jointly optimizing these objectives, DMAE learns temporally coherent and semantically rich representations. Comprehensive evaluations across classification, regression, and forecasting tasks demonstrate that our approach achieves consistent and superior performance over competitive baselines.

Method: ReaSyn explores synthesizable space by generating synthetic pathways using chain-of-reaction (CoR) notation that explicitly states reactants, reaction types, and intermediate products. It uses supervised training with dense supervision and reinforcement learning-based finetuning with test-time compute scaling.

Result: ReaSyn achieves the highest reconstruction rate, pathway diversity in synthesizable molecule reconstruction, and highest optimization performance in goal-directed molecular optimization, significantly outperforming previous methods in synthesizable hit expansion.

Conclusion: ReaSyn demonstrates superior ability to navigate combinatorially-large synthesizable chemical space through its novel CoR approach and reasoning-based framework.

Abstract: A well-known pitfall of molecular generative models is that they are not guaranteed to generate synthesizable molecules. There have been considerable attempts to address this problem, but given the exponentially large combinatorial space of synthesizable molecules, existing methods have shown limited coverage of the space and poor molecular optimization performance. To tackle these problems, we introduce ReaSyn, a generative framework for synthesizable projection where the model explores the neighborhood of given molecules in the synthesizable space by generating pathways that result in synthesizable analogs. To fully utilize the chemical knowledge contained in the synthetic pathways, we propose a novel perspective that views synthetic pathways akin to reasoning paths in large language models (LLMs). Specifically, inspired by chain-of-thought (CoT) reasoning in LLMs, we introduce the chain-of-reaction (CoR) notation that explicitly states reactants, reaction types, and intermediate products for each step in a pathway. With the CoR notation, ReaSyn can get dense supervision in every reaction step to explicitly learn chemical reaction rules during supervised training and perform step-by-step reasoning. In addition, to further enhance the reasoning capability of ReaSyn, we propose reinforcement learning (RL)-based finetuning and goal-directed test-time compute scaling tailored for synthesizable projection. ReaSyn achieves the highest reconstruction rate and pathway diversity in synthesizable molecule reconstruction and the highest optimization performance in synthesizable goal-directed molecular optimization, and significantly outperforms previous synthesizable projection methods in synthesizable hit expansion. These results highlight ReaSyn’s superior ability to navigate combinatorially-large synthesizable chemical space.

Method: DiffusionNFT optimizes diffusion models directly on the forward process via flow matching, contrasting positive and negative generations to define an implicit policy improvement direction that incorporates reinforcement signals into supervised learning.

Result: DiffusionNFT is up to 25x more efficient than FlowGRPO, improves GenEval score from 0.24 to 0.98 within 1k steps (vs. FlowGRPO’s 0.95 with 5k+ steps and CFG), and significantly boosts SD3.5-Medium performance across benchmarks.

Conclusion: DiffusionNFT provides a more efficient and flexible RL paradigm for diffusion models that eliminates the need for likelihood estimation and classifier-free guidance while achieving superior performance.

Abstract: Online reinforcement learning (RL) has been central to post-training language models, but its extension to diffusion models remains challenging due to intractable likelihoods. Recent works discretize the reverse sampling process to enable GRPO-style training, yet they inherit fundamental drawbacks, including solver restrictions, forward-reverse inconsistency, and complicated integration with classifier-free guidance (CFG). We introduce Diffusion Negative-aware FineTuning (DiffusionNFT), a new online RL paradigm that optimizes diffusion models directly on the forward process via flow matching. DiffusionNFT contrasts positive and negative generations to define an implicit policy improvement direction, naturally incorporating reinforcement signals into the supervised learning objective. This formulation enables training with arbitrary black-box solvers, eliminates the need for likelihood estimation, and requires only clean images rather than sampling trajectories for policy optimization. DiffusionNFT is up to $25\times$ more efficient than FlowGRPO in head-to-head comparisons, while being CFG-free. For instance, DiffusionNFT improves the GenEval score from 0.24 to 0.98 within 1k steps, while FlowGRPO achieves 0.95 with over 5k steps and additional CFG employment. By leveraging multiple reward models, DiffusionNFT significantly boosts the performance of SD3.5-Medium in every benchmark tested.

Method: The method connects generative outputs to oracle classification tasks (e.g., classifying responses as harmful/harmless or clustering semantically equivalent answers). It develops theory that associates the number of RS samples with robustness radius, assuming bounded error rates for oracle classifiers.

Result: The paper derives improved scaling laws analytically relating certified radius and accuracy to sample count, showing that 2-3 orders of magnitude fewer samples suffice with minimal loss even under weaker assumptions.

Conclusion: These advances make robustness certification both well-defined and computationally feasible for state-of-the-art VLMs, as validated against jailbreak-style adversarial attacks.

Abstract: Randomized smoothing (RS) is one of the prominent techniques to ensure the correctness of machine learning models, where point-wise robustness certificates can be derived analytically. While RS is well understood for classification, its application to generative models is unclear, since their outputs are sequences rather than labels. We resolve this by connecting generative outputs to an oracle classification task and showing that RS can still be enabled: the final response can be classified as a discrete action (e.g., service-robot commands in VLAs), as harmful vs. harmless (content moderation or toxicity detection in VLMs), or even applying oracles to cluster answers into semantically equivalent ones. Provided that the error rate for the oracle classifier comparison is bounded, we develop the theory that associates the number of samples with the corresponding robustness radius. We further derive improved scaling laws analytically relating the certified radius and accuracy to the number of samples, showing that the earlier result of 2 to 3 orders of magnitude fewer samples sufficing with minimal loss remains valid even under weaker assumptions. Together, these advances make robustness certification both well-defined and computationally feasible for state-of-the-art VLMs, as validated against recent jailbreak-style adversarial attacks.

Method: Developed GANet framework using genetic algorithm-based network optimization with PageRank and Degree metrics to analyze 159 salivary samples via ATR-FTIR spectroscopy, systematically optimizing network structure for pattern extraction from high-dimensional spectral data.

Result: GANet achieved 0.78 accuracy, 0.61 sensitivity, 0.90 specificity, and 0.74 harmonic mean, outperforming linear discriminant analysis, support vector machines, and deep learning models.

Conclusion: GANet demonstrates strong potential as a robust, bio-inspired, non-invasive tool for precise ASD detection and broader spectral-based health applications.

Abstract: Autism Spectrum Disorder (ASD) lacks reliable biological markers, delaying early diagnosis. Using 159 salivary samples analyzed by ATR-FTIR spectroscopy, we developed GANet, a genetic algorithm-based network optimization framework leveraging PageRank and Degree for importance-based feature characterization. GANet systematically optimizes network structure to extract meaningful patterns from high-dimensional spectral data. It achieved superior performance compared to linear discriminant analysis, support vector machines, and deep learning models, reaching 0.78 accuracy, 0.61 sensitivity, 0.90 specificity, and a 0.74 harmonic mean. These results demonstrate GANet’s potential as a robust, bio-inspired, non-invasive tool for precise ASD detection and broader spectral-based health applications.

Method: Integrates heat-kernel coefficients from quantum field theory with Tucker and CP decompositions, using matriculation/vectorization techniques for hidden structure discovery. Employs dual-level optimization: local heat-kernel enhanced fuzzy clustering with tensor decomposition, and federated aggregation of tensor factors with differential privacy.

Result: Enables efficient handling of high-dimensional multi-view data with significant communication savings through low-rank tensor approximations and privacy-preserving mechanisms.

Conclusion: The proposed tensorized framework provides an effective solution for personalized federated learning with enhanced privacy protection and computational efficiency for complex multi-view data structures.

Abstract: We present a robust personalized federated learning framework that leverages heat-kernel enhanced tensorized multi-view fuzzy c-means clustering with advanced tensor decomposition techniques. Our approach integrates heat-kernel coefficients adapted from quantum field theory with Tucker decomposition and canonical polyadic decomposition (CANDECOMP/PARAFAC) to transform conventional distance metrics and efficiently represent high-dimensional multi-view structures. The framework employs matriculation and vectorization techniques to facilitate the discovery of hidden structures and multilinear relationships via N-way generalized tensors. The proposed method introduces a dual-level optimization scheme: local heat-kernel enhanced fuzzy clustering with tensor decomposition operating on order-N input tensors, and federated aggregation of tensor factors with privacy-preserving personalization mechanisms. The local stage employs tensorized kernel Euclidean distance transformations and Tucker decomposition to discover client-specific patterns in multi-view tensor data, while the global aggregation process coordinates tensor factors (core tensors and factor matrices) across clients through differential privacy-preserving protocols. This tensorized approach enables efficient handling of high-dimensional multi-view data with significant communication savings through low-rank tensor approximations.

Method: Leverages online feedback from latent-space evaluations (CLIP, discriminator, human preference model) to perform greedy search for optimal CFG scale at each diffusion timestep, creating prompt-specific guidance schedules.

Result: Achieves up to 53.8% human preference win-rate overall and 55.5% on text rendering prompts compared to Imagen 3 baseline, with improvements in text alignment, visual quality, and numerical reasoning.

Conclusion: Optimal guidance schedules are inherently dynamic and prompt-dependent, and the framework provides an efficient, generalizable solution for adaptive CFG scaling.

Abstract: Classifier-free guidance (CFG) is a cornerstone of text-to-image diffusion models, yet its effectiveness is limited by the use of static guidance scales. This “one-size-fits-all” approach fails to adapt to the diverse requirements of different prompts; moreover, prior solutions like gradient-based correction or fixed heuristic schedules introduce additional complexities and fail to generalize. In this work, we challeng this static paradigm by introducing a framework for dynamic CFG scheduling. Our method leverages online feedback from a suite of general-purpose and specialized small-scale latent-space evaluations, such as CLIP for alignment, a discriminator for fidelity and a human preference reward model, to assess generation quality at each step of the reverse diffusion process. Based on this feedback, we perform a greedy search to select the optimal CFG scale for each timestep, creating a unique guidance schedule tailored to every prompt and sample. We demonstrate the effectiveness of our approach on both small-scale models and the state-of-the-art Imagen 3, showing significant improvements in text alignment, visual quality, text rendering and numerical reasoning. Notably, when compared against the default Imagen 3 baseline, our method achieves up to 53.8% human preference win-rate for overall preference, a figure that increases up to to 55.5% on prompts targeting specific capabilities like text rendering. Our work establishes that the optimal guidance schedule is inherently dynamic and prompt-dependent, and provides an efficient and generalizable framework to achieve it.

Method: A multi-field spatio-temporal deep learning model that integrates pressure, density, temperature, energy, material distribution, and velocity components into an autoregressive surrogate trained on high-fidelity hydrocode data.

Result: MSTM achieves 1000x speedup over direct simulation with errors below 4% in porous materials and below 10% in lattice structures, while preserving integrated quantities like mass-averaged pressure and temperature within 5%.

Conclusion: The model transforms previously intractable problems into tractable design studies, enabling optimization of meso-structured materials for planetary impact mitigation, fusion energy, and security applications.

Abstract: The ability to predict how shock waves traverse porous and architected materials is a decisive factor in planetary defense, national security, and the race to achieve inertial fusion energy. Yet capturing pore collapse, anomalous Hugoniot responses, and localized heating – phenomena that can determine the success of asteroid deflection or fusion ignition – has remained a major challenge despite recent advances in single-field and reduced representations. We introduce a multi-field spatio-temporal deep learning model (MSTM) that unifies seven coupled fields – pressure, density, temperature, energy, material distribution, and two velocity components – into a single autoregressive surrogate. Trained on high-fidelity hydrocode data, MSTM runs about a thousand times faster than direct simulation, achieving errors below 4% in porous materials and below 10% in lattice structures. Unlike prior single-field or operator-based surrogates, MSTM resolves sharp shock fronts while preserving integrated quantities such as mass-averaged pressure and temperature to within 5%. This advance transforms problems once considered intractable into tractable design studies, establishing a practical framework for optimizing meso-structured materials in planetary impact mitigation, inertial fusion energy, and national security.

Method: Frames automated cyber defense as a two-player context-based partially observable Markov decision problem using attributed graph representations. Agents learn through relational inductive bias, treating actions as edits to the environmental graph.

Result: The approach significantly outperforms state-of-the-art methods and enables agents to defend never-before-seen networks against various adversaries in complex, multi-agent environments.

Conclusion: Graph-based representations with relational inductive bias provide superior generalization capabilities for cyber defense agents, allowing zero-shot adaptation to new network topologies and improved performance across diverse scenarios.

Abstract: Deep reinforcement learning (RL) is emerging as a viable strategy for automated cyber defense (ACD). The traditional RL approach represents networks as a list of computers in various states of safety or threat. Unfortunately, these models are forced to overfit to specific network topologies, rendering them ineffective when faced with even small environmental perturbations. In this work, we frame ACD as a two-player context-based partially observable Markov decision problem with observations represented as attributed graphs. This approach allows our agents to reason through the lens of relational inductive bias. Agents learn how to reason about hosts interacting with other system entities in a more general manner, and their actions are understood as edits to the graph representing the environment. By introducing this bias, we will show that our agents can better reason about the states of networks and zero-shot adapt to new ones. We show that this approach outperforms the state-of-the-art by a wide margin, and makes our agents capable of defending never-before-seen networks against a wide range of adversaries in a variety of complex, and multi-agent environments.

Method: DiveBatch uses a data-driven adaptation based on gradient diversity, which has strong theoretical justification from SGD convergence analysis. The algorithm dynamically adjusts batch size to maintain small-batch generalization benefits while improving computational efficiency.

Result: Evaluations on synthetic data, CIFAR-10, CIFAR-100, and Tiny-ImageNet show DiveBatch converges 1.06-5.0x faster than standard SGD and AdaBatch, with only a slight performance trade-off.

Conclusion: DiveBatch successfully addresses the batch size adaptation challenge, achieving significant speed improvements while preserving generalization performance, making it an effective approach for accelerating deep learning training.

Abstract: The goal of this paper is to accelerate the training of machine learning models, a critical challenge since the training of large-scale deep neural models can be computationally expensive. Stochastic gradient descent (SGD) and its variants are widely used to train deep neural networks. In contrast to traditional approaches that focus on tuning the learning rate, we propose a novel adaptive batch size SGD algorithm, DiveBatch, that dynamically adjusts the batch size. Adapting the batch size is challenging: using large batch sizes is more efficient due to parallel computation, but small-batch training often converges in fewer epochs and generalizes better. To address this challenge, we introduce a data-driven adaptation based on gradient diversity, enabling DiveBatch to maintain the generalization performance of small-batch training while improving convergence speed and computational efficiency. Gradient diversity has a strong theoretical justification: it emerges from the convergence analysis of SGD. Evaluations of DiveBatch on synthetic and CiFar-10, CiFar-100, and Tiny-ImageNet demonstrate that DiveBatch converges significantly faster than standard SGD and AdaBatch (1.06 – 5.0x), with a slight trade-off in performance.

Method: Three-component approach: 1) Discrete search with greedy trigger accretion from singletons, 2) Implicit blacklisting using cosine similarity in activation space, 3) Detection based on high misclassification rates with unusual confidence.

Result: The method reliably detects and successfully inverts ground-truth backdoor trigger phrases, unlike many recent approaches.

Conclusion: The proposed approach effectively addresses LLM-specific challenges in backdoor detection and provides a practical solution for trigger inversion in discrete input spaces.

Abstract: While effective backdoor detection and inversion schemes have been developed for AIs used e.g. for images, there are challenges in “porting” these methods to LLMs. First, the LLM input space is discrete, which precludes gradient-based search over this space, central to many backdoor inversion methods. Second, there are ~30,000^k k-tuples to consider, k the token-length of a putative trigger. Third, for LLMs there is the need to blacklist tokens that have strong marginal associations with the putative target response (class) of an attack, as such tokens give false detection signals. However, good blacklists may not exist for some domains. We propose a LLM trigger inversion approach with three key components: i) discrete search, with putative triggers greedily accreted, starting from a select list of singletons; ii) implicit blacklisting, achieved by evaluating the average cosine similarity, in activation space, between a candidate trigger and a small clean set of samples from the putative target class; iii) detection when a candidate trigger elicits high misclassifications, and with unusually high decision confidence. Unlike many recent works, we demonstrate that our approach reliably detects and successfully inverts ground-truth backdoor trigger phrases.

Method: Proposes GraphSTAD system combining convolutional neural networks (for local spatial features), graph neural networks (for global channel connections), and recurrent neural networks (for temporal evolution) on 3D digi-occupancy map data.

Result: The system achieves production-level accuracy in capturing diverse channel fault types using LHC collision data, and is being integrated into CMS core production system for real-time HCAL monitoring.

Conclusion: GraphSTAD demonstrates promising performance compared to benchmark models and provides an effective solution for real-time anomaly detection in high-energy physics experiments.

Abstract: The Compact Muon Solenoid (CMS) experiment is a general-purpose detector for high-energy collision at the Large Hadron Collider (LHC) at CERN. It employs an online data quality monitoring (DQM) system to promptly spot and diagnose particle data acquisition problems to avoid data quality loss. In this study, we present a semi-supervised spatio-temporal anomaly detection (AD) monitoring system for the physics particle reading channels of the Hadron Calorimeter (HCAL) of the CMS using three-dimensional digi-occupancy map data of the DQM. We propose the GraphSTAD system, which employs convolutional and graph neural networks to learn local spatial characteristics induced by particles traversing the detector and the global behavior owing to shared backend circuit connections and housing boxes of the channels, respectively. Recurrent neural networks capture the temporal evolution of the extracted spatial features. We validate the accuracy of the proposed AD system in capturing diverse channel fault types using the LHC collision data sets. The GraphSTAD system achieves production-level accuracy and is being integrated into the CMS core production system for real-time monitoring of the HCAL. We provide a quantitative performance comparison with alternative benchmark models to demonstrate the promising leverage of the presented system. Code: https://github.com/muleina/CMS_HCAL_ML_OnlineDQM .

Method: Con$_2$ consists of two components: Context Contrasting (clustering representations by context) and Content Alignment (aligning normal sample positions across clusters), leveraging symmetries in normal data to observe samples in different contexts.

Result: The method outperforms competitive baselines on specialized medical datasets and shows competitive performance on natural imaging benchmarks, demonstrating effective anomaly detection without prior anomaly knowledge.

Conclusion: Con$_2$ provides a novel approach to anomaly detection by using symmetry-based contextual learning, eliminating the need for prior anomaly knowledge while achieving strong performance across diverse datasets.

Abstract: Anomaly detection focuses on identifying samples that deviate from the norm. Discovering informative representations of normal samples is crucial to detecting anomalies effectively. Recent self-supervised methods have successfully learned such representations by employing prior knowledge about anomalies to create synthetic outliers during training. However, we often do not know what to expect from unseen data in specialized real-world applications. In this work, we address this limitation with our new approach Con$_2$, which leverages prior knowledge about symmetries in normal samples to observe the data in different contexts. Con$_2$ consists of two parts: Context Contrasting clusters representations according to their context, while Content Alignment encourages the model to capture semantic information by aligning the positions of normal samples across clusters. The resulting representation space allows us to detect anomalies as outliers of the learned context clusters. We demonstrate the benefit of this approach in extensive experiments on specialized medical datasets, outperforming competitive baselines based on self-supervised learning and pretrained models and presenting competitive performance on natural imaging benchmarks.

Method: Proposes Directed Random Walk (DiRW) with direction-aware path sampling optimized for walk probability, length, and number in a weight-free manner. Uses node-wise learnable path aggregator for generalized node representations.

Result: Extensive experiments on 9 datasets show DiRW enhances most spatial-based methods as plug-and-play strategy and achieves state-of-the-art performance as new digraph learning paradigm.

Conclusion: DiRW provides an effective solution for directed graph learning that overcomes limitations of existing approaches and demonstrates superior performance across multiple datasets.

Abstract: Recently, graph neural network (GNN) has emerged as a powerful representation learning tool for graph-structured data. However, most approaches are tailored for undirected graphs, neglecting the abundant information in the edges of directed graphs (digraphs). In fact, digraphs are widely applied in the real world and confirmed to address heterophily challenges. Despite recent advancements, existing spatial- and spectral-based DiGNNs have limitations due to their complex learning mechanisms and reliance on high-quality topology, resulting in low efficiency and unstable performance. To address these issues, we propose Directed Random Walk (DiRW), a plug-and-play strategy for most spatial-based DiGNNs and also an innovative model which offers a new digraph learning paradigm. Specifically, it utilizes a direction-aware path sampler optimized from the perspectives of walk probability, length, and number in a weight-free manner by considering node profiles and topologies. Building upon this, DiRW incorporates a node-wise learnable path aggregator for generalized node representations. Extensive experiments on 9 datasets demonstrate that DiRW: (1) enhances most spatial-based methods as a plug-and-play strategy; (2) achieves SOTA performance as a new digraph learning paradigm. The source code and data are available at https://github.com/dhsiuu/DiRW.

Method: BC-LLM iteratively searches over potentially infinite concept sets within a Bayesian framework, using LLMs as both concept extraction mechanism and prior. Despite LLM limitations like miscalibration and hallucinations, the method provides rigorous statistical inference and uncertainty quantification.

Result: Across image, text, and tabular datasets, BC-LLM outperforms interpretable baselines and even black-box models in certain settings, converges more rapidly towards relevant concepts, and demonstrates greater robustness to out-of-distribution samples.

Conclusion: The proposed BC-LLM framework successfully addresses limitations of traditional CBMs by leveraging LLMs in a Bayesian setting, achieving better performance while maintaining interpretability and providing statistical guarantees.

Abstract: Concept Bottleneck Models (CBMs) have been proposed as a compromise between white-box and black-box models, aiming to achieve interpretability without sacrificing accuracy. The standard training procedure for CBMs is to predefine a candidate set of human-interpretable concepts, extract their values from the training data, and identify a sparse subset as inputs to a transparent prediction model. However, such approaches are often hampered by the tradeoff between exploring a sufficiently large set of concepts versus controlling the cost of obtaining concept extractions, resulting in a large interpretability-accuracy tradeoff. This work investigates a novel approach that sidesteps these challenges: BC-LLM iteratively searches over a potentially infinite set of concepts within a Bayesian framework, in which Large Language Models (LLMs) serve as both a concept extraction mechanism and prior. Even though LLMs can be miscalibrated and hallucinate, we prove that BC-LLM can provide rigorous statistical inference and uncertainty quantification. Across image, text, and tabular datasets, BC-LLM outperforms interpretable baselines and even black-box models in certain settings, converges more rapidly towards relevant concepts, and is more robust to out-of-distribution samples.

Method: Built cross-temporal hierarchies by leveraging both cross-sectional (turbine-level) and temporal hierarchical structures in wind farms, using machine learning-based forecasts with cross-temporal reconciliation.

Result: Cross-temporal reconciliation was superior to individual cross-sectional reconciliation at multiple temporal aggregations, with high accuracy at coarser temporal granularities.

Conclusion: Cross-temporal hierarchical reconciliation provides valuable insights for decision-makers on optimal methods for forecasting high-frequency wind data across different horizons and levels.

Abstract: Renewable energy generation is of utmost importance for global decarbonization. Forecasting renewable energies, particularly wind energy, is challenging due to the inherent uncertainty in wind energy generation, which depends on weather conditions. Recent advances in hierarchical forecasting through reconciliation have demonstrated a significant increase in the quality of wind energy forecasts for short-term periods. We leverage the cross-sectional and temporal hierarchical structure of turbines in wind farms and build cross-temporal hierarchies to further investigate how integrated cross-sectional and temporal dimensions can add value to forecast accuracy in wind farms. We found that cross-temporal reconciliation was superior to individual cross-sectional reconciliation at multiple temporal aggregations. Additionally, machine learning based forecasts that were cross-temporally reconciled demonstrated high accuracy at coarser temporal granularities, which may encourage adoption for short-term wind forecasts. Empirically, we provide insights for decision-makers on the best methods for forecasting high-frequency wind data across different forecasting horizons and levels.

Method: Theoretical analysis of gradient conflicts, demonstration that second-order optimization naturally resolves conflicts through implicit gradient alignment, and introduction of SOAP - a quasi-Newton method that efficiently approximates Hessian preconditioner. Also introduces a novel gradient alignment score generalizing cosine similarity to multiple gradients.

Result: Breakthrough performance in PINNs: state-of-the-art results on 10 challenging PDE benchmarks, including first successful application to turbulent flows with Reynolds numbers up to 10,000, achieving 2-10x accuracy improvements over existing methods.

Conclusion: Establishes frameworks for understanding and resolving gradient conflicts with broad implications for optimization beyond scientific computing, demonstrating that second-order methods effectively address fundamental challenges in multi-task learning.

Abstract: Multi-task learning through composite loss functions is fundamental to modern deep learning, yet optimizing competing objectives remains challenging. We present new theoretical and practical approaches for addressing directional conflicts between loss terms, demonstrating their effectiveness in physics-informed neural networks (PINNs) where such conflicts are particularly challenging to resolve. Through theoretical analysis, we demonstrate how these conflicts limit first-order methods and show that second-order optimization naturally resolves them through implicit gradient alignment. We prove that SOAP, a recently proposed quasi-Newton method, efficiently approximates the Hessian preconditioner, enabling breakthrough performance in PINNs: state-of-the-art results on 10 challenging PDE benchmarks, including the first successful application to turbulent flows with Reynolds numbers up to 10,000, with 2-10x accuracy improvements over existing methods. We also introduce a novel gradient alignment score that generalizes cosine similarity to multiple gradients, providing a practical tool for analyzing optimization dynamics. Our findings establish frameworks for understanding and resolving gradient conflicts, with broad implications for optimization beyond scientific computing.

Method: Implementing a negotiation paradigm where models negotiate output representations with previously determined class labels, tested on CIFAR-10, CIFAR-100, and MNIST datasets in overfitting scenarios.

Result: The approach increased average classification accuracy and decreased overfitting rates without using other regularization techniques, demonstrating broader applicability beyond its intended purpose.

Conclusion: The negotiation paradigm shows promise for overcoming learning challenges and should be explored further by the machine learning community, particularly in areas like continual learning.

Abstract: Overfitting is a phenomenon that occurs when a machine learning model is trained for too long and focused too much on the exact fitness of the training samples to the provided training labels and cannot keep track of the predictive rules that would be useful on the test data. This phenomenon is commonly attributed to memorization of particular samples, memorization of the noise, and forced fitness into a data set of limited samples by using a high number of neurons. While it is true that the model encodes various peculiarities as the training process continues, we argue that most of the overfitting occurs in the process of reconciling sharply defined membership ratios. In this study, we present an approach that increases the classification accuracy of machine learning models by allowing the model to negotiate output representations of the samples with previously determined class labels. By setting up a negotiation between the models interpretation of the inputs and the provided labels, we not only increased average classification accuracy but also decreased the rate of overfitting without applying any other regularization tricks. By implementing our negotiation paradigm approach to several low regime machine learning problems by generating overfitting scenarios from publicly available data sets such as CIFAR 10, CIFAR 100, and MNIST we have demonstrated that the proposed paradigm has more capacity than its intended purpose. We are sharing the experimental results and inviting the machine learning community to explore the limits of the proposed paradigm. We also aim to incentive the community to exploit the negotiation paradigm to overcome the learning related challenges in other research fields such as continual learning. The Python code of the experimental setup is uploaded to GitHub.

Method: Probabilistic Adaptive Performance Estimation (PAPE) evaluates binary classification models on unlabeled tabular data by estimating performance under covariate shift. It works independently of the original model, using only predictions and probability estimates, learns directly from data without assumptions about shift nature, and applies to any confusion matrix-based metric.

Result: Tested on over 900 dataset-model combinations from US census data, PAPE outperformed several benchmarks across various metrics.

Conclusion: PAPE is a superior choice for estimating binary classification model performance, providing accurate performance estimation under covariate shift without requiring labeled data or assumptions about the shift.

Abstract: After deployment, machine learning models often experience performance degradation due to shifts in data distribution. It is challenging to assess post-deployment performance accurately when labels are missing or delayed. Existing proxy methods, such as data drift detection, fail to measure the effects of these shifts adequately. To address this, we introduce a new method for evaluating binary classification models on unlabeled tabular data that accurately estimates model performance under covariate shift and call it Probabilistic Adaptive Performance Estimation (PAPE). It can be applied to any performance metric defined with elements of the confusion matrix. Crucially, PAPE operates independently of the original model, relying only on its predictions and probability estimates, and does not need any assumptions about the nature of covariate shift, learning directly from data instead. We tested PAPE using over 900 dataset-model combinations from US census data, assessing its performance against several benchmarks through various metrics. Our findings show that PAPE outperforms other methodologies, making it a superior choice for estimating the performance of binary classification models.

Method: Uses small neural networks (InfluenceNetwork) that are only 0.0027% the size of full language models to estimate influence values, applied to subset selection for instruction fine-tuning.

Result: Achieves up to 99% cost reduction compared to traditional methods while showing no performance compromise with four state-of-the-art influence functions.

Conclusion: NN-CIFT provides an efficient and scalable alternative to traditional influence function calculations, enabling practical influence analysis for large language models.

Abstract: Influence functions provide crucial insights into model training, but existing methods suffer from large computational costs and limited generalization. Particularly, recent works have proposed various metrics and algorithms to calculate the influence of data using language models, which do not scale well with large models and datasets. This is because of the expensive forward and backward passes required for computation, substantial memory requirements to store large models, and poor generalization of influence estimates to new data. In this paper, we explore the use of small neural networks – which we refer to as the InfluenceNetwork – to estimate influence values, achieving up to 99% cost reduction. Our evaluation demonstrates that influence values can be estimated with models just 0.0027% the size of full language models (we use 7B and 8B versions). We apply our algorithm of estimating influence values (called NN-CIFT: Neural Networks for effiCient Instruction Fine-Tuning) to the downstream task of subset selection for general instruction fine-tuning. In our study, we include four state-of-the-art influence functions and show no compromise in performance, despite large speedups, between NN-CIFT and the original influence functions. We provide an in-depth hyperparameter analyses of NN-CIFT. The code for our method can be found here: https://github.com/agarwalishika/NN-CIFT.

Method: The authors derive necessary and sufficient constraints on exponential family LVM parameters that ensure prior and posterior distributions over latent variables belong to the same exponential family, enabling exact inference algorithms.

Result: The theory identifies various well-known and novel models that satisfy these constraints, and the authors develop generalized inference and learning algorithms for these LVMs.

Conclusion: This unified framework facilitates understanding and implementing exact inference for diverse models, potentially guiding researchers toward discovering new models that avoid approximation schemes.

Abstract: Bayes’ rule describes how to infer posterior beliefs about latent variables given observations, and inference is a critical step in learning algorithms for latent variable models (LVMs). Although there are exact algorithms for inference and learning for certain LVMs such as linear Gaussian models and mixture models, researchers must typically develop approximate inference and learning algorithms when applying novel LVMs. Here we study the line that separates LVMs that rely on approximation schemes from those that do not, and develop a general theory of exponential family LVMs for which inference and learning may be implemented exactly. Firstly, under mild assumptions about the exponential family form of the LVM, we derive a necessary and sufficient constraint on the parameters of the LVM under which the prior and posterior over the latent variables are in the same exponential family. We then show that a variety of well-known and novel models indeed have this constrained, exponential family form. Finally, we derive generalized inference and learning algorithms for these LVMs, and demonstrate them with a variety of examples. Our unified perspective facilitates both understanding and implementing exact inference and learning algorithms for a wide variety of models, and may guide researchers in the discovery of new models that avoid unnecessary approximations.

Method: Proposes FedAdam-SSM which sparsifies local model parameters and moment estimates using a shared sparse mask (SSM) instead of three separate masks. Theoretically optimizes the SSM to minimize divergence from centralized Adam by considering sparsification error and data distribution imbalance.

Result: FedAdam-SSM achieves over 1.1x faster convergence than sparse FedAdam baselines and over 14.5% higher test accuracy than quantized FedAdam baselines. The method effectively reduces communication overhead while maintaining performance.

Conclusion: FedAdam-SSM successfully addresses the communication overhead problem in federated Adam algorithms through intelligent sparsification with shared masks, providing both theoretical guarantees and practical performance improvements in federated learning settings.

Abstract: Adaptive moment estimation (Adam), as a Stochastic Gradient Descent (SGD) variant, has gained widespread popularity in federated learning (FL) due to its fast convergence. However, federated Adam (FedAdam) algorithms suffer from a threefold increase in uplink communication overhead compared to federated SGD (FedSGD) algorithms, which arises from the necessity to transmit both local model updates and first and second moment estimates from distributed devices to the centralized server for aggregation. Driven by this issue, we propose a novel sparse FedAdam algorithm called FedAdam-SSM, wherein distributed devices sparsify the updates of local model parameters and moment estimates and subsequently upload the sparse representations to the centralized server. To further reduce the communication overhead, the updates of local model parameters and moment estimates incorporate a shared sparse mask (SSM) into the sparsification process, eliminating the need for three separate sparse masks. Theoretically, we develop an upper bound on the divergence between the local model trained by FedAdam-SSM and the desired model trained by centralized Adam, which is related to sparsification error and imbalanced data distribution. By minimizing the divergence bound between the model trained by FedAdam-SSM and centralized Adam, we optimize the SSM to mitigate the learning performance degradation caused by sparsification error. Additionally, we provide convergence bounds for FedAdam-SSM in both convex and non-convex objective function settings, and investigate the impact of local epoch, learning rate and sparsification ratio on the convergence rate of FedAdam-SSM. Experimental results show that FedAdam-SSM outperforms baselines in terms of convergence rate (over 1.1$\times$ faster than the sparse FedAdam baselines) and test accuracy (over 14.5% ahead of the quantized FedAdam baselines).

Method: Temporal-Feature Cross Attention Mechanism (TFCAM) - a novel deep learning framework inspired by transformer architectures that captures dynamic interactions among clinical features across time.

Result: Outperformed LSTM and RETAIN baselines in predicting End-Stage Renal Disease progression in 1,422 CKD patients, achieving AUROC of 0.95 and F1-score of 0.69.

Conclusion: TFCAM successfully addresses deep learning’s black box limitations in healthcare by providing multi-level explainability while maintaining state-of-the-art predictive performance, offering clinicians transparent insights into disease progression mechanisms.

Abstract: Despite the outstanding performance of deep learning models in clinical prediction tasks, explainability remains a significant challenge. Inspired by transformer architectures, we introduce the Temporal-Feature Cross Attention Mechanism (TFCAM), a novel deep learning framework designed to capture dynamic interactions among clinical features across time, enhancing both predictive accuracy and interpretability. In an experiment with 1,422 patients with Chronic Kidney Disease, predicting progression to End-Stage Renal Disease, TFCAM outperformed LSTM and RETAIN baselines, achieving an AUROC of 0.95 and an F1-score of 0.69. Beyond performance gains, TFCAM provides multi-level explainability by identifying critical temporal periods, ranking feature importance, and quantifying how features influence each other across time before affecting predictions. Our approach addresses the “black box” limitations of deep learning in healthcare, offering clinicians transparent insights into disease progression mechanisms while maintaining state-of-the-art predictive performance.

Method: Proposes TDAlign framework with two innovations: 1) a loss function aligning change values between adjacent time steps in predictions with target patterns, and 2) an adaptive loss balancing strategy that integrates with existing methods without additional parameters.

Result: Extensive experiments show TDAlign reduces baseline prediction errors by 1.47% to 9.19% and change value errors by 4.57% to 15.78% across six baselines and seven datasets.

Conclusion: TDAlign effectively enhances existing LTSF methods with minimal computational overhead, demonstrating substantial performance improvements in temporal dependency modeling.

Abstract: Long-term time series forecasting (LTSF) is a critical task across diverse domains. Despite significant advancements in LTSF research, we identify a performance bottleneck in existing LTSF methods caused by the inadequate modeling of Temporal Dependencies within the Target (TDT). To address this issue, we propose a novel and generic temporal modeling framework, Temporal Dependency Alignment (TDAlign), that equips existing LTSF methods with TDT learning capabilities. TDAlign introduces two key innovations: 1) a loss function that aligns the change values between adjacent time steps in the predictions with those in the target, ensuring consistency with variation patterns, and 2) an adaptive loss balancing strategy that seamlessly integrates the new loss function with existing LTSF methods without introducing additional learnable parameters. As a plug-and-play framework, TDAlign enhances existing methods with minimal computational overhead, featuring only linear time complexity and constant space complexity relative to the prediction length. Extensive experiments on six strong LTSF baselines across seven real-world datasets demonstrate the effectiveness and flexibility of TDAlign. On average, TDAlign reduces baseline prediction errors by \textbf{1.47%} to \textbf{9.19%} and change value errors by \textbf{4.57%} to \textbf{15.78%}, highlighting its substantial performance improvements.

Method: FRIDA utilizes membership and property inference attacks to directly infer evidence of genuine client training, rather than focusing on implicit effects of free-riding.

Result: Extensive evaluation demonstrates that FRIDA is effective across a wide range of scenarios.

Conclusion: The proposed FRIDA method provides an effective solution for detecting free-riders in federated learning systems using privacy attack techniques.

Abstract: Federated learning is increasingly popular as it enables multiple parties with limited datasets and resources to train a machine learning model collaboratively. However, similar to other collaborative systems, federated learning is vulnerable to free-riders - participants who benefit from the global model without contributing. Free-riders compromise the integrity of the learning process and slow down the convergence of the global model, resulting in increased costs for honest participants. To address this challenge, we propose FRIDA: free-rider detection using privacy attacks. Instead of focusing on implicit effects of free-riding, FRIDA utilizes membership and property inference attacks to directly infer evidence of genuine client training. Our extensive evaluation demonstrates that FRIDA is effective across a wide range of scenarios.

Method: Proposes a noise-corrected version of Langevin algorithm that removes bias from noisy score functions, with a special case involving iterative noise addition and half-noise removal.

Result: The method successfully removes first-order bias terms when using noisy score functions for sampling.

Conclusion: The proposed algorithm enables effective sampling using noisy score functions with minimal requirements, offering an intuitive iterative noise manipulation approach.

Abstract: The Langevin algorithm is a classic method for sampling from a given pdf in a real space. In its basic version, it only requires knowledge of the gradient of the log-density, also called the score function. However, in deep learning, it is often easier to learn the so-called “noisy-data score function”, i.e. the gradient of the log-density of noisy data, more precisely when Gaussian noise is added to the data. Such an estimate is biased and complicates the use of the Langevin method. Here, we propose a noise-corrected version of the Langevin algorithm, where the bias due to noisy data is removed, at least regarding first-order terms. Unlike diffusion models, our algorithm only needs to know the noisy score function for one single noise level. We further propose a simple special case which has an interesting intuitive interpretation of iteratively adding noise the data and then attempting to remove half of that noise.

Method: Analyzed hydrology time series from CAMELS and Caravan global datasets using 8 different model configurations to assess impact of exogenous data. Compared over 20 state-of-the-art pattern and foundation models with LSTM-based modeling and data preprocessing via Jupyter Notebook on Google Colab.

Result: Integrating exogenous information enhances data representation, reducing mean squared error by up to 40% in the largest dataset. Models incorporating comprehensive observed and exogenous data outperform limited approaches, including foundation models.

Conclusion: Natural annual periodic exogenous time series contribute the most significant improvements, though static and other periodic factors are also valuable. The research provides an open-source framework for scientific time series analysis.

Abstract: There has been active investigation into deep learning approaches for time series analysis, including foundation models. However, most studies do not address significant scientific applications. This paper aims to identify key features in time series by examining hydrology data. Our work advances computer science by emphasizing critical application features and contributes to hydrology and other scientific fields by identifying modeling approaches that effectively capture these features. Scientific time series data are inherently complex, involving observations from multiple locations, each with various time-dependent data streams and exogenous factors that may be static or time-varying and either application-dependent or purely mathematical. This research analyzes hydrology time series from the CAMELS and Caravan global datasets, which encompass rainfall and runoff data across catchments, featuring up to six observed streams and 209 static parameters across approximately 8,000 locations. Our investigation assesses the impact of exogenous data through eight different model configurations for key hydrology tasks. Results demonstrate that integrating exogenous information enhances data representation, reducing mean squared error by up to 40% in the largest dataset. Additionally, we present a detailed performance comparison of over 20 state-of-the-art pattern and foundation models. The analysis is fully open-source, facilitated by Jupyter Notebook on Google Colab for LSTM-based modeling, data preprocessing, and model comparisons. Preliminary findings using alternative deep learning architectures reveal that models incorporating comprehensive observed and exogenous data outperform more limited approaches, including foundation models. Notably, natural annual periodic exogenous time series contribute the most significant improvements, though static and other periodic factors are also valuable.

Method: ConCISE identifies two redundant reflection patterns (Confidence Deficit and Termination Delay) and uses Confidence Injection to boost reasoning confidence plus Early Stopping to terminate reasoning when confidence is sufficient.

Result: Fine-tuning LRMs on ConCISE-generated data reduces reasoning chain length by up to ~50% while maintaining high task accuracy, achieving better compression-performance balance than baseline methods.

Conclusion: ConCISE effectively addresses verbose reasoning in LRMs through confidence-guided compression, significantly reducing computational overhead without sacrificing reasoning quality.

Abstract: Large Reasoning Models (LRMs) perform strongly in complex reasoning tasks via Chain-of-Thought (CoT) prompting, but often suffer from verbose outputs, increasing computational overhead. Existing fine-tuning-based compression methods either operate post-hoc pruning, risking disruption to reasoning coherence, or rely on sampling-based selection, which fails to remove redundant content thoroughly. To address these limitations, this work begins by framing two key patterns of redundant reflection in LRMs–Confidence Deficit, wherein the model reflects on correct intermediate steps, and Termination Delay, where reflection continues after a verified, confident answer–through a confidence-guided perspective. Based on this, we introduce ConCISE (Confidence-guided Compression In Step-by-step Efficient Reasoning), a framework designed to generate concise reasoning chains, integrating Confidence Injection to boost reasoning confidence, and Early Stopping to terminate reasoning when confidence is sufficient. Extensive experiments demonstrate that compared to baseline methods, fine-tuning LRMs on ConCISE-generated data yields a better balance between compression and task performance, reducing length by up to approximately 50% under SimPO, while maintaining high task accuracy.

Method: Systematic, data-driven analysis of research trends from 1994-2024, with comparative evaluation of multimodal data fusion approaches, data types, fusion techniques, and remote sensing platforms.

Result: Transformers outperform conventional models by enhancing prediction accuracy, mitigating feature redundancy, and optimizing large-scale data integration for soil analysis, crop classification, yield prediction, and disease detection.

Conclusion: The review provides a structured roadmap and strategic framework for implementing data fusion in agricultural remote sensing, offering best practices for ground-truth data selection, platform integration, and fusion model design to advance precision agriculture.

Abstract: This review explores recent advancements in data fusion techniques and Transformer-based remote sensing applications in precision agriculture. Using a systematic, data-driven approach, we analyze research trends from 1994 to 2024, identifying key developments in data fusion, remote sensing, and AI-driven agricultural monitoring. While traditional machine learning and deep learning approaches have demonstrated effectiveness in agricultural decision-making, challenges such as limited scalability, suboptimal feature extraction, and reliance on extensive labeled data persist. This study examines the comparative advantages of Transformer-based fusion methods, particularly their ability to model spatiotemporal dependencies and integrate heterogeneous datasets for applications in soil analysis, crop classification, yield prediction, and disease detection. A comparative analysis of multimodal data fusion approaches is conducted, evaluating data types, fusion techniques, and remote sensing platforms. We demonstrate how Transformers outperform conventional models by enhancing prediction accuracy, mitigating feature redundancy, and optimizing large-scale data integration. Furthermore, we propose a structured roadmap for implementing data fusion in agricultural remote sensing, outlining best practices for ground-truth data selection, platform integration, and fusion model design. By addressing key research gaps and providing a strategic framework, this review offers valuable insights for advancing precision agriculture through AI-driven data fusion techniques.

Method: Studied steepest descent algorithms with infinitesimal learning rate in deep homogeneous neural networks, analyzing training trajectories and their connections to margin-maximization KKT conditions.

Result: Found that geometric margin increases after networks achieve perfect training accuracy, and limit points of training trajectories correspond to KKT points of margin-maximization problems. Experimental results show connections to Adam and Shampoo optimizers.

Conclusion: Steepest descent algorithms exhibit implicit bias toward margin maximization in deep homogeneous networks, with optimization trajectories converging to solutions that satisfy KKT conditions for margin maximization.

Abstract: We study the implicit bias of the general family of steepest descent algorithms with infinitesimal learning rate in deep homogeneous neural networks. We show that: (a) an algorithm-dependent geometric margin starts increasing once the networks reach perfect training accuracy, and (b) any limit point of the training trajectory corresponds to a KKT point of the corresponding margin-maximization problem. We experimentally zoom into the trajectories of neural networks optimized with various steepest descent algorithms, highlighting connections to the implicit bias of popular adaptive methods (Adam and Shampoo).

Method: Schrier-Coset Graph Propagation (SCGP) enriches node features through Schreier-coset embeddings without altering input graph topology, embedding bottleneck-free connectivity patterns into compact feature space.

Result: SCGP achieves performance comparable to or exceeding expander graph and rewired GNN baselines across node and graph classification benchmarks, with particular advantages on hierarchical/modular structures.

Conclusion: SCGP offers reduced inference latency, improved scalability, and low memory footprint, making it suitable for real-time and resource-constrained applications while effectively addressing over-squashing.

Abstract: Graph Neural Networks (GNNs) offer a principled framework for learning over graph-structured data, yet their expressive capacity is often hindered by over-squashing, wherein information from distant nodes is compressed into fixed-size vectors. Existing solutions, including graph rewiring and bottleneck-resistant architectures such as Cayley and expander graphs, avoid this problem but introduce scalability bottlenecks. In particular, the Cayley graphs constructed over $SL(2,\mathbb{Z}_n)$ exhibit strong theoretical properties, yet suffer from cubic node growth $O(n^3)$, leading to high memory usage. To address this, this work introduces Schrier-Coset Graph Propagation (SCGP), a group-theoretic augmentation method that enriches node features through Schreier-coset embeddings without altering the input graph topology. SCGP embeds bottleneck-free connectivity patterns into a compact feature space, improving long-range message passing while maintaining computational efficiency. Empirical evaluations across standard node and graph classification benchmarks demonstrate that SCGP achieves performance comparable to, or exceeding, expander graph and rewired GNN baselines. Furthermore, SCGP exhibits particular advantages in processing hierarchical and modular graph structures, offering reduced inference latency, improved scalability, and a low memory footprint, making it suitable for real-time and resource-constrained applications.

Method: Proposes ER-PRM that integrates KL-regularized Markov Decision Processes to balance policy optimization while preventing excessive policy shift. Derives novel reward construction method based on theoretical analysis showing optimal reward model can be derived from initial policy sampling.

Result: ER-PRM consistently outperforms existing process reward models, achieving 1% improvement on GSM8K and 2-3% improvement on MATH under best-of-N evaluation, and over 1% improvement under RLHF.

Conclusion: Entropy-regularization effectively enhances LLMs’ reasoning capabilities, demonstrating the efficacy of the proposed ER-PRM approach in mathematical reasoning tasks.

Abstract: Large language models (LLMs) have shown promise in performing complex multi-step reasoning, yet they continue to struggle with mathematical reasoning, often making systematic errors. A promising solution is reinforcement learning (RL) guided by reward models, particularly those focusing on process rewards, which score each intermediate step rather than solely evaluating the final outcome. This approach is more effective at guiding policy models towards correct reasoning trajectories. In this work, we propose an entropy-regularized process reward model (ER-PRM) that integrates KL-regularized Markov Decision Processes (MDP) to balance policy optimization with the need to prevent the policy from shifting too far from its initial distribution. We derive a novel reward construction method based on the theoretical results. Our theoretical analysis shows that we could derive the optimal reward model from the initial policy sampling. Our empirical experiments on the MATH and GSM8K benchmarks demonstrate that ER-PRM consistently outperforms existing process reward models, achieving 1% improvement on GSM8K and 2-3% improvement on MATH under best-of-N evaluation, and more than 1% improvement under RLHF. These results highlight the efficacy of entropy-regularization in enhancing LLMs’ reasoning capabilities.

Method: Style Encoder Adversarial Domain Adaptation (SE-ADA) separates domain-specific information from low-dimensional representations and minimizes domain differences using adversarial learning.

Result: SE-ADA effectively removed domain information while preserving brain structure features and achieved the highest disease search accuracy compared to recent domain harmonization methods on eight public brain MR datasets.

Conclusion: The proposed SE-ADA method successfully addresses domain gap issues in multi-site brain MR studies and enables more accurate content-based image retrieval by preserving pathological features while harmonizing domain differences.

Abstract: When conducting large-scale studies that collect brain MR images from multiple facilities, the impact of differences in imaging equipment and protocols at each site cannot be ignored, and this domain gap has become a significant issue in recent years. In this study, we propose a new low-dimensional representation (LDR) acquisition method called style encoder adversarial domain adaptation (SE-ADA) to realize content-based image retrieval (CBIR) of brain MR images. SE-ADA reduces domain differences while preserving pathological features by separating domain-specific information from LDR and minimizing domain differences using adversarial learning. In evaluation experiments comparing SE-ADA with recent domain harmonization methods on eight public brain MR datasets (ADNI1/2/3, OASIS1/2/3/4, PPMI), SE-ADA effectively removed domain information while preserving key aspects of the original brain structure and demonstrated the highest disease search accuracy.

Method: The paper conducts a comprehensive survey reviewing both traditional solutions and modern LLM-driven approaches for each of the four graph learning challenges, analyzing how LLMs provide unique advantages.

Result: The survey demonstrates that LLMs can effectively address graph learning challenges by leveraging their semantic understanding capabilities, though the field remains emerging with open research questions.

Conclusion: LLMs show promising potential for overcoming fundamental graph learning challenges, representing an important interdisciplinary research direction that requires further exploration and development.

Abstract: Graphs are a widely used paradigm for representing non-Euclidean data, with applications ranging from social network analysis to biomolecular prediction. While graph learning has achieved remarkable progress, real-world graph data presents a number of challenges that significantly hinder the learning process. In this survey, we focus on four fundamental data-centric challenges: (1) Incompleteness, real-world graphs have missing nodes, edges, or attributes; (2) Imbalance, the distribution of the labels of nodes or edges and their structures for real-world graphs are highly skewed; (3) Cross-domain Heterogeneity, graphs from different domains exhibit incompatible feature spaces or structural patterns; and (4) Dynamic Instability, graphs evolve over time in unpredictable ways. Recently, Large Language Models (LLMs) offer the potential to tackle these challenges by leveraging rich semantic reasoning and external knowledge. This survey focuses on how LLMs can address four fundamental data-centric challenges in graph-structured data, thereby improving the effectiveness of graph learning. For each challenge, we review both traditional solutions and modern LLM-driven approaches, highlighting how LLMs contribute unique advantages. Finally, we discuss open research questions and promising future directions in this emerging interdisciplinary field. To support further exploration, we have curated a repository of recent advances on graph learning challenges: https://github.com/limengran98/Awesome-Literature-Graph-Learning-Challenges.

Method: Theoretical analysis of RSGD convergence with different batch size strategies (constant vs increasing), using principal component analysis and low-rank matrix completion for numerical validation.

Result: Increasing batch size leads to faster convergence than constant batch size, reduces SFO complexity, and combines advantages of both small and large constant batch sizes.

Conclusion: Increasing batch size strategy is superior to constant batch size for RSGD, providing improved convergence rates and computational efficiency.

Abstract: We theoretically analyzed the convergence behavior of Riemannian stochastic gradient descent (RSGD) and found that using an increasing batch size leads to faster convergence than using a constant batch size, not only with a constant learning rate but also with a decaying learning rate, such as cosine annealing decay and polynomial decay. The convergence rate improves from $O(T^{-1}+C)$ with a constant batch size to $O(T^{-1})$ with an increasing batch size, where $T$ denotes the total number of iterations and $C$ is a constant. Using principal component analysis and low-rank matrix completion, we investigated, both theoretically and numerically, how an increasing batch size affects computational time as quantified by stochastic first-order oracle (SFO) complexity. An increasing batch size was found to reduce the SFO complexity of RSGD. Furthermore, an increasing batch size was found to offer the advantages of both small and large constant batch sizes.

Method: The paper proposes a Residual Learning algorithm that solves a bilevel optimization problem to provably converge to critical points, addressing issues caused by asymmetric response functions and other hardware imperfections like limited granularity.

Result: Theoretical analysis shows asymmetric response functions impose an implicit penalty on the objective in Analog SGD, and the proposed Residual Learning method can overcome these issues with provable convergence guarantees.

Conclusion: This is the first paper to systematically investigate the impact of generic non-ideal response functions in AIMC training, with simulations validating the theoretical insights and demonstrating the effectiveness of the proposed Residual Learning approach.

Abstract: As the economic and environmental costs of training and deploying large vision or language models increase dramatically, analog in-memory computing (AIMC) emerges as a promising energy-efficient solution. However, the training perspective, especially its training dynamic, is underexplored. In AIMC hardware, the trainable weights are represented by the conductance of resistive elements and updated using consecutive electrical pulses. While the conductance changes by a constant in response to each pulse, in reality, the change is scaled by asymmetric and non-linear \textit{response functions}, leading to a non-ideal training dynamic. This paper provides a theoretical foundation for gradient-based training on AIMC hardware with non-ideal response functions. We demonstrate that asymmetric response functions negatively impact Analog SGD by imposing an implicit penalty on the objective. To overcome the issue, we propose Residual Learning algorithm, which provably converges exactly to a critical point by solving a bilevel optimization problem. We demonstrate that the proposed method can be extended to address other hardware imperfections, such as limited response granularity. As we know, it is the first paper to investigate the impact of a class of generic non-ideal response functions. The conclusion is supported by simulations validating our theoretical insights.

Method: Proposes LoPE framework with asymmetric LoRA configuration featuring a dedicated poisoning expert. Uses two-stage paradigm: noise injection on poisoning expert during fine-tuning to enhance noise discrimination, and selective masking of poisoning expert during inference to leverage purified knowledge from normal experts.

Result: Extensive experiments show LoPE achieves strong performance and robustness through low-cost noise injection, completely eliminating data cleaning requirements.

Conclusion: LoPE provides an effective noise-robust adaptation method that enhances model robustness to noise using only generated noisy data, outperforming conventional approaches.

Abstract: Current parameter-efficient fine-tuning methods for adapting pre-trained language models to downstream tasks are susceptible to interference from noisy data. Conventional noise-handling approaches either rely on laborious data pre-processing or employ model architecture modifications prone to error accumulation. In contrast to existing noise-process paradigms, we propose a noise-robust adaptation method via asymmetric LoRA poisoning experts (LoPE), a novel framework that enhances model robustness to noise only with generated noisy data. Drawing inspiration from the mixture-of-experts architecture, LoPE strategically integrates a dedicated poisoning expert in an asymmetric LoRA configuration. Through a two-stage paradigm, LoPE performs noise injection on the poisoning expert during fine-tuning to enhance its noise discrimination and processing ability. During inference, we selectively mask the dedicated poisoning expert to leverage purified knowledge acquired by normal experts for noise-robust output. Extensive experiments demonstrate that LoPE achieves strong performance and robustness purely through the low-cost noise injection, which completely eliminates the requirement of data cleaning.

Method: The survey investigates fixed, learnable, relative, and hybrid positional encoding approaches, evaluating their effectiveness based on data characteristics like sequence length, signal complexity, and dimensionality.

Result: Findings show that data characteristics significantly influence method effectiveness, with advanced positional encoding methods providing performance gains in prediction accuracy but at the cost of increased computational complexity.

Conclusion: The survey provides a comprehensive overview and quantitative benchmarking to help researchers and practitioners select and design effective positional encoding methods for transformer-based time series models, while outlining key challenges and future research directions.

Abstract: Recent advancements in transformer-based models have greatly improved time series analysis, providing robust solutions for tasks such as forecasting, anomaly detection, and classification. A crucial element of these models is positional encoding, which allows transformers to capture the intrinsic sequential nature of time series data. This survey systematically examines existing techniques for positional encoding in transformer-based time series models. We investigate a variety of methods, including fixed, learnable, relative, and hybrid approaches, and evaluate their effectiveness in different time series classification tasks. Our findings indicate that data characteristics like sequence length, signal complexity, and dimensionality significantly influence method effectiveness. Advanced positional encoding methods exhibit performance gains in terms of prediction accuracy, however, they come at the cost of increased computational complexity. Furthermore, we outline key challenges and suggest potential research directions to enhance positional encoding strategies. By delivering a comprehensive overview and quantitative benchmarking, this survey intends to assist researchers and practitioners in selecting and designing effective positional encoding methods for transformer-based time series models.

Method: Train a single non-linear multi-label classifier on hidden activations and compute intervention directions via gradients at inference time, enabling dynamic composition of behaviors without retraining.

Result: K-Steering outperforms strong baselines across 3 model families on new benchmarks (ToneBank and DebateMix), validated by both activation-based classifiers and LLM-based judges.

Conclusion: The proposed K-Steering method provides a flexible, unified approach for multi-attribute behavioral control in LLMs, avoiding linearity assumptions and eliminating the need for storing/tuning separate attribute vectors.

Abstract: Controlling multiple behavioral attributes in large language models (LLMs) at inference time is a challenging problem due to interference between attributes and the limitations of linear steering methods, which assume additive behavior in activation space and require per-attribute tuning. We introduce K-Steering, a unified and flexible approach that trains a single non-linear multi-label classifier on hidden activations and computes intervention directions via gradients at inference time. This avoids linearity assumptions, removes the need for storing and tuning separate attribute vectors, and allows dynamic composition of behaviors without retraining. To evaluate our method, we propose two new benchmarks, ToneBank and DebateMix, targeting compositional behavioral control. Empirical results across 3 model families, validated by both activation-based classifiers and LLM-based judges, demonstrate that K-Steering outperforms strong baselines in accurately steering multiple behaviors.

Method: CoT uses large VLMs (master) to cache high-quality query results, then employs multi-modal retrieval and in-context learning to help smaller VLMs (apprentice) by retrieving relevant cached responses during inference.

Result: CoT increases overall reasoning performance by up to 7.7% under the same budget and boosts apprentice VLM performance by up to 36.6% on challenging general reasoning benchmarks.

Conclusion: The proposed Cache of Thought framework effectively bridges the performance gap between large and small VLMs, enabling cost-efficient deployment while maintaining high reasoning quality through collaborative inference.

Abstract: Vision Language Models (VLMs) have achieved remarkable success in a wide range of vision applications of increasing complexity and scales, yet choosing the right VLM model size involves a trade-off between response quality and cost. While smaller VLMs are cheaper to run, they typically produce responses only marginally better than random guessing on benchmarks such as MMMU. In this paper, we propose Cache of Thought (CoT), a master apprentice framework for collaborative inference between large and small VLMs. CoT manages high quality query results from large VLMs (master) in a cache, which are then selected via a novel multi modal retrieval and in-context learning to aid the performance of small VLMs (apprentice). We extensively evaluate CoT on various widely recognized and challenging general reasoning benchmarks, and show that CoT increases overall reasoning performance by up to 7.7% under the same budget, and specifically boosts the performance of apprentice VLMs by up to 36.6%. Our code is available at https://github.com/UIUC-MONET/Cache-of-Thoughts

Method: Uses generative adversarial networks (GANs) to construct explanation graphs similar to original graphs, with a novel objective function for the generator that includes dynamic loss weight scheme to maximize prediction probability for specific classes.

Result: Experimental results show GIN-Graph can interpret GNNs trained on various graph datasets and generate high-quality explanation graphs with high stability and reliability.

Conclusion: GIN-Graph successfully addresses the limitations of existing model-level interpretation methods by producing reliable and high-quality explanation graphs for GNNs using a GAN-based approach.

Abstract: One significant challenge of exploiting Graph neural networks (GNNs) in real-life scenarios is that they are always treated as black boxes, therefore leading to the requirement of interpretability. To address this, model-level interpretation methods have been developed to explain what patterns maximize probability of predicting to a certain class. However, existing model-level interpretation methods pose several limitations such as generating invalid explanation graphs and lacking reliability. In this paper, we propose a new Generative Interpretation Network for Model-Level Explanation of Graph Neural Networks (GIN-Graph), to generate reliable and high-quality model-level explanation graphs. The implicit and likelihood-free generative adversarial networks are exploited to construct the explanation graphs which are similar to original graphs, meanwhile maximizing the prediction probability for a certain class by adopting a novel objective function for generator with dynamic loss weight scheme. Experimental results indicate that GIN-Graph can be applied to interpret GNNs trained on a variety of graph datasets and generate high-quality explanation graphs with high stability and reliability.

Method: Proposes Perception-R1 with a novel visual perception reward that uses textual visual annotations from CoT trajectories. A judging LLM assesses consistency between annotations and MLLM responses to assign rewards during RLVR training.

Result: Extensive experiments show Perception-R1 achieves state-of-the-art performance on multiple multimodal reasoning benchmarks using only 1,442 training data.

Conclusion: The proposed visual perception reward effectively enhances both multimodal perception and reasoning capabilities of MLLMs, addressing limitations of existing RLVR methods.

Abstract: Enhancing the multimodal reasoning capabilities of Multimodal Large Language Models (MLLMs) is a challenging task that has attracted increasing attention in the community. Recently, several studies have applied Reinforcement Learning with Verifiable Rewards (RLVR) to the multimodal domain in order to enhance the reasoning abilities of MLLMs. However, these works largely overlook the enhancement of multimodal perception capabilities in MLLMs, which serve as a core prerequisite and foundational component of complex multimodal reasoning. Through McNemar’s test, we find that existing RLVR method fails to effectively enhance the multimodal perception capabilities of MLLMs, thereby limiting their further improvement in multimodal reasoning. To address this limitation, we propose Perception-R1, which introduces a novel visual perception reward that explicitly encourages MLLMs to perceive the visual content accurately, thereby can effectively incentivizing both their multimodal perception and reasoning capabilities. Specifically, we first collect textual visual annotations from the CoT trajectories of multimodal problems, which will serve as visual references for reward assignment. During RLVR training, we employ a judging LLM to assess the consistency between the visual annotations and the responses generated by MLLM, and assign the visual perception reward based on these consistency judgments. Extensive experiments on several multimodal reasoning benchmarks demonstrate the effectiveness of our Perception-R1, which achieves state-of-the-art performance on most benchmarks using only 1,442 training data.

Method: Autoregressive Spatio-temporal Fourier Transformer with dual-path architecture integrating frequency-domain and spatio-temporal representations, hierarchical blocks for multi-scale dynamics capture, and generative residual correction for probabilistic refinement.

Result: The approach demonstrates advantages over state-of-the-art ML methods on three benchmark datasets (plasma, fluid, and atmospheric dynamics), showing improved accuracy and reliability in long-term probabilistic forecasting.

Conclusion: The proposed StFT framework effectively addresses long-term forecasting challenges in multi-scale systems by explicitly capturing cross-scale dynamics and providing robust uncertainty quantification through generative residual correction.

Abstract: Simulating the long-term dynamics of multi-scale and multi-physics systems poses a significant challenge in understanding complex phenomena across science and engineering. The complexity arises from the intricate interactions between scales and the interplay of diverse physical processes, which manifest in PDEs through coupled, nonlinear terms that govern the evolution of multiple physical fields across scales. Neural operators have shown potential in short-term prediction of such complex spatio-temporal dynamics; however, achieving stable high-fidelity predictions and providing robust uncertainty quantification over extended time horizons remains an open and unsolved area of research. These limitations often lead to stability degradation with rapid error accumulation, particularly in long-term forecasting of systems characterized by multi-scale behaviors involving dynamics of different orders. To address these challenges, we propose an autoregressive Spatio-temporal Fourier Transformer (StFT), in which each transformer block is designed to learn the system dynamics at a distinct scale through a dual-path architecture that integrates frequency-domain and spatio-temporal representations. By leveraging a structured hierarchy of \ours blocks, the resulting model explicitly captures the underlying dynamics across both macro- and micro- spatial scales. Furthermore, a generative residual correction mechanism is introduced to learn a probabilistic refinement temporally while simultaneously quantifying prediction uncertainties, enhancing both the accuracy and reliability of long-term probabilistic forecasting. Evaluations conducted on three benchmark datasets (plasma, fluid, and atmospheric dynamics) demonstrate the advantages of our approach over state-of-the-art ML methods.

Method: Conceptual analysis and empirical illustrations to examine limitations of existing fairness measures, challenging prevailing views on accuracy-fairness incompatibility and incompatibility among fairness measures.

Result: Demonstrates limited applicability of existing fairness measures in complex real-world scenarios, challenging common misconceptions about fairness incompatibilities.

Conclusion: Outlines three principles for designing better fairness measures to bridge the gap between technical formalization and social realities, helping meet real-world AI/ML deployment challenges.

Abstract: The rapid trend of deploying artificial intelligence (AI) and machine learning (ML) systems in socially consequential domains has raised growing concerns about their trustworthiness, including potential discriminatory behaviours. Research in algorithmic fairness has generated a proliferation of mathematical definitions and metrics, yet persistent misconceptions and limitations – both within and beyond the fairness community – limit their effectiveness, such as an unreached consensus on its understanding, prevailing measures primarily tailored to binary group settings, and superficial handling for intersectional contexts. Here we critically remark on these misconceptions and argue that fairness cannot be reduced to purely technical constraints on models; we also examine the limitations of existing fairness measures through conceptual analysis and empirical illustrations, showing their limited applicability in the face of complex real-world scenarios, challenging prevailing views on the incompatibility between accuracy and fairness as well as that among fairness measures themselves, and outlining three worth-considering principles in the design of fairness measures. We believe these findings will help bridge the gap between technical formalisation and social realities and meet the challenges of real-world AI/ML deployment.

Method: Proposes direct analytics on time series data compressed using the SHRINK compression algorithm, using outlier detection as a case study to demonstrate the approach.

Result: Outperforms baselines on uncompressed data in multiple cases (with only 1% difference in worst case), achieves 4x lower runtime on average, and accesses only 10% of data volume.

Conclusion: The approach enables reliable, high-speed outlier detection for IoT applications while achieving high compression, reducing data transmission, and supporting edge analytics with limited resources.

Abstract: Semantic communication has emerged as a promising paradigm to tackle the challenges of massive growing data traffic and sustainable data communication. It shifts the focus from data fidelity to goal-oriented or task-oriented semantic transmission. While deep learning-based methods are commonly used for semantic encoding and decoding, they struggle with the sequential nature of time series data and high computation cost, particularly in resource-constrained IoT environments. Data compression plays a crucial role in reducing transmission and storage costs, yet traditional data compression methods fall short of the demands of goal-oriented communication systems. In this paper, we propose a novel method for direct analytics on time series data compressed by the SHRINK compression algorithm. Through experimentation using outlier detection as a case study, we show that our method outperforms baselines running on uncompressed data in multiple cases, with merely 1% difference in the worst case. Additionally, it achieves four times lower runtime on average and accesses approximately 10% of the data volume, which enables edge analytics with limited storage and computation power. These results demonstrate that our approach offers reliable, high-speed outlier detection analytics for diverse IoT applications while extracting semantics from time-series data, achieving high compression, and reducing data transmission.

Method: PRISM (Polysemantic FeatuRe Identification and Scoring Method) is introduced as a framework specifically designed to capture feature complexity in LLMs. Unlike single-description-per-neuron approaches, PRISM produces nuanced descriptions that account for both monosemantic and polysemantic behavior.

Result: Through extensive benchmarking against existing methods, PRISM demonstrates more accurate and faithful feature descriptions, improving both overall description quality (via description score) and the ability to capture distinct concepts when polysemanticity is present (via polysemanticity score).

Conclusion: PRISM successfully addresses the limitations of current automated interpretability methods by providing a framework that better captures the complexity of features in LLMs, particularly handling polysemantic neuron behavior more effectively.

Abstract: Automated interpretability research aims to identify concepts encoded in neural network features to enhance human understanding of model behavior. Within the context of large language models (LLMs) for natural language processing (NLP), current automated neuron-level feature description methods face two key challenges: limited robustness and the assumption that each neuron encodes a single concept (monosemanticity), despite increasing evidence of polysemanticity. This assumption restricts the expressiveness of feature descriptions and limits their ability to capture the full range of behaviors encoded in model internals. To address this, we introduce Polysemantic FeatuRe Identification and Scoring Method (PRISM), a novel framework specifically designed to capture the complexity of features in LLMs. Unlike approaches that assign a single description per neuron, common in many automated interpretability methods in NLP, PRISM produces more nuanced descriptions that account for both monosemantic and polysemantic behavior. We apply PRISM to LLMs and, through extensive benchmarking against existing methods, demonstrate that our approach produces more accurate and faithful feature descriptions, improving both overall description quality (via a description score) and the ability to capture distinct concepts when polysemanticity is present (via a polysemanticity score).

Method: The authors conduct a systematic review by tracing historical development, formalizing mathematical frameworks, categorizing modeling methods, analyzing training/inference techniques, and discussing applications across language, vision-language, and biological domains.

Result: The survey shows that d(M)LLMs achieve performance comparable to autoregressive models while enabling up to 10× acceleration in inference speed, positioning them as promising alternatives to traditional approaches.

Conclusion: Discrete diffusion models represent a significant advancement in language modeling paradigms, with growing industrial and academic adoption, and the paper identifies future research directions for continued development and deployment.

Abstract: In this work, we provide a systematic survey of Discrete Diffusion Language Models (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs). Unlike autoregressive (AR) models, dLLMs and dMLLMs adopt a multi-token, parallel decoding paradigm using full attention and a denoising-based generation strategy. This paradigm naturally enables parallel generation, fine-grained output control, and dynamic perception. These capabilities are previously difficult to achieve with AR models. A growing number of industrial-scale proprietary d(M)LLMs, as well as a large number of open-source academic d(M)LLMs, have demonstrated performance comparable to their autoregressive counterparts, while achieving up to 10$\times$ acceleration in inference speed. These developments position discrete diffusion models as a promising alternative to intelligence based on the traditional autoregressive approach. In this work, we present a comprehensive overview of the research in the dLLM and dMLLM domains. We trace the historical development of dLLMs and dMLLMs, formalize the underlying mathematical frameworks, list commonly-used modeling methods, and categorize representative models. We further analyze key techniques for training, inference, quantization. We also discuss the trustworthy issues and summarize emerging applications across language, vision-language, and biological domains and etc.. We conclude by discussing future directions for research and deployment. Relative papers are collected in https://github.com/LiQiiiii/Awesome-Discrete-Diffusion-LLM_MLLM

Method: SPACE maximizes consistency of spike-behavior-based local feature maps across augmented versions of a single test sample, enabling adaptation without requiring source data.

Result: SPACE outperforms state-of-the-art ANN methods across multiple datasets and diverse network architectures (CNNs, Transformer, ConvLSTM) while maintaining lower computational cost.

Conclusion: SPACE demonstrates effective and robust adaptation for SNNs in real-world settings, highlighting its practical applicability for energy-efficient neuromorphic computing.

Abstract: Spiking Neural Networks (SNNs), as a biologically plausible alternative to Artificial Neural Networks (ANNs), have demonstrated advantages in terms of energy efficiency, temporal processing, and biological plausibility. However, SNNs are highly sensitive to distribution shifts, which can significantly degrade their performance in real-world scenarios. Traditional test-time adaptation (TTA) methods designed for ANNs often fail to address the unique computational dynamics of SNNs, such as sparsity and temporal spiking behavior. To address these challenges, we propose SPike-Aware Consistency Enhancement (SPACE), the first source-free and single-instance TTA method specifically designed for SNNs. SPACE leverages the inherent spike dynamics of SNNs to maximize the consistency of spike-behavior-based local feature maps across augmented versions of a single test sample, enabling robust adaptation without requiring source data. We evaluate SPACE on multiple datasets. Furthermore, SPACE exhibits robust generalization across diverse network architectures, consistently enhancing the performance of SNNs on CNNs, Transformer, and ConvLSTM architectures. Experimental results show that SPACE outperforms state-of-the-art ANN methods while maintaining lower computational cost, highlighting its effectiveness and robustness for SNNs in real-world settings. The code will be available at https://github.com/ethanxyluo/SPACE.

Method: Extends the generalized Projected Bellman Error objective to support multistep credit assignment via λ-returns, deriving three gradient-based methods with both forward-view (compatible with experience replay) and backward-view (compatible with streaming) formulations.

Result: The proposed algorithms outperform both PPO and StreamQ in MuJoCo and MinAtar environments, demonstrating improved performance and stability.

Conclusion: The multistep GTD methods provide stable and efficient off-policy learning with better credit assignment capabilities, bridging the gap between theoretical convergence guarantees and practical deep RL performance.

Abstract: Achieving fast and stable off-policy learning in deep reinforcement learning (RL) is challenging. Most existing methods rely on semi-gradient temporal-difference (TD) methods for their simplicity and efficiency, but are consequently susceptible to divergence. While more principled approaches like Gradient TD (GTD) methods have strong convergence guarantees, they have rarely been used in deep RL. Recent work introduced the generalized Projected Bellman Error ($\overline{\text{PBE}}$), enabling GTD methods to work efficiently with nonlinear function approximation. However, this work is limited to one-step methods, which are slow at credit assignment and require a large number of samples. In this paper, we extend the generalized $\overline{\text{PBE}}$ objective to support multistep credit assignment based on the $\lambda$-return and derive three gradient-based methods that optimize this new objective. We provide both a forward-view formulation compatible with experience replay and a backward-view formulation compatible with streaming algorithms. Finally, we evaluate the proposed algorithms and show that they outperform both PPO and StreamQ in MuJoCo and MinAtar environments, respectively. Code available at https://github.com/esraaelelimy/gtd_algos

Method: Frequency domain representation combined with a 1-dimensional convolutional neural network for compound fault diagnosis, tested on the PHM Beijing 2024 dataset with 21 sensor channels and faults from 4 interacting components.

Result: Achieved 97.67% accuracy on test set with 17 single faults and 93.93% accuracy on test set with 42 compound faults.

Conclusion: The proposed method effectively addresses compound fault diagnosis in interacting train transmission components, demonstrating superior performance compared to existing single-fault models.

Abstract: Fault diagnosis prevents train disruptions by ensuring the stability and reliability of their transmission systems. Data-driven fault diagnosis models have several advantages over traditional methods in terms of dealing with non-linearity, adaptability, scalability, and automation. However, existing data-driven models are trained on separate transmission components and only consider single faults due to the limitations of existing datasets. These models will perform worse in scenarios where components operate with each other at the same time, affecting each component’s vibration signals. To address some of these challenges, we propose a frequency domain representation and a 1-dimensional convolutional neural network for compound fault diagnosis and applied it on the PHM Beijing 2024 dataset, which includes 21 sensor channels, 17 single faults, and 42 compound faults from 4 interacting components, that is, motor, gearbox, left axle box, and right axle box. Our proposed model achieved 97.67% and 93.93% accuracies on the test set with 17 single faults and on the test set with 42 compound faults, respectively.

Method: Contextualized, task-relevant (CTR) attention networks that generate attention maps from both human and RL agent gameplay in Atari environments, validated against temporally integrated overt attention (TIOA) models from eye-tracking data.

Result: Human CTR maps are more sparse than agent maps and align better with TIOA maps, suggesting they capture internal attention patterns. When used to guide RL agents, they achieve slightly improved and more stable learning.

Conclusion: The work provides a new approach for extracting and validating internal attention from behavioral data, advancing understanding of human-agent attention differences and enabling better RL agent guidance.

Abstract: This study introduces a novel method for revealing human internal attention patterns from gameplay data alone, leveraging offline attention techniques from reinforcement learning (RL). We propose contextualized, task-relevant (CTR) attention networks, which generate attention maps from both human and RL agent gameplay in Atari environments. To evaluate whether the human CTR maps reveal internal attention, we validate our model by quantitative and qualitative comparison to the agent maps as well as to a temporally integrated overt attention (TIOA) model based on human eye-tracking data. Our results show that human CTR maps are more sparse than the agent ones and align better with the TIOA maps. Following a qualitative visual comparison we conclude that they likely capture patterns of internal attention. As a further application, we use these maps to guide RL agents, finding that human internal attention-guided agents achieve slightly improved and more stable learning compared to baselines. This work advances the understanding of human-agent attention differences and provides a new approach for extracting and validating internal attention from behavioral data.

Method: Two-stage approach: 1) CAN-U model with IPM and propensity score regularization generates complete dataset with counterfactual uplift labels; 2) CAN-D model with isotonic output layer directly models uplift effects without regularization, adaptively correcting CAN-U errors through factual reinforcement. Context-Aware Attention Layer manages interactions between treatment, merchant, and contextual features.

Result: Extensive experiments on two real-world datasets validate TSCAN’s effectiveness. Deployment on one of China’s largest online food ordering platforms demonstrates practical utility and impact for real-world merchant diagnosis.

Conclusion: TSCAN successfully addresses limitations of traditional ITE estimation methods by eliminating regularization dependencies while maintaining bias mitigation, and effectively models varying treatment effects across different contexts through context-aware attention mechanisms.

Abstract: A primary challenge in ITE estimation is sample selection bias. Traditional approaches utilize treatment regularization techniques such as the Integral Probability Metrics (IPM), re-weighting, and propensity score modeling to mitigate this bias. However, these regularizations may introduce undesirable information loss and limit the performance of the model. Furthermore, treatment effects vary across different external contexts, and the existing methods are insufficient in fully interacting with and utilizing these contextual features. To address these issues, we propose a Context-Aware uplift model based on the Two-Stage training approach (TSCAN), comprising CAN-U and CAN-D sub-models. In the first stage, we train an uplift model, called CAN-U, which includes the treatment regularizations of IPM and propensity score prediction, to generate a complete dataset with counterfactual uplift labels. In the second stage, we train a model named CAN-D, which utilizes an isotonic output layer to directly model uplift effects, thereby eliminating the reliance on the regularization components. CAN-D adaptively corrects the errors estimated by CAN-U through reinforcing the factual samples, while avoiding the negative impacts associated with the aforementioned regularizations. Additionally, we introduce a Context-Aware Attention Layer throughout the two-stage process to manage the interactions between treatment, merchant, and contextual features, thereby modeling the varying treatment effect in different contexts. We conduct extensive experiments on two real-world datasets to validate the effectiveness of TSCAN. Ultimately, the deployment of our model for real-world merchant diagnosis on one of China’s largest online food ordering platforms validates its practical utility and impact.

Method: Gaussian Process-based policy iteration with alternating policy evaluation and update steps, incorporating additive Schwarz acceleration as preconditioning

Result: Numerical experiments show Schwarz acceleration improves computational efficiency

Conclusion: The proposed framework effectively solves HJB equations and MFGs with enhanced convergence through Schwarz acceleration

Abstract: We propose a Gaussian Process (GP)-based policy iteration framework for addressing both forward and inverse problems in Hamilton–Jacobi–Bellman (HJB) equations and mean field games (MFGs). Policy iteration is formulated as an alternating procedure between solving the value function under a fixed control policy and updating the policy based on the resulting value function. By exploiting the linear structure of GPs for function approximation, each policy evaluation step admits an explicit closed-form solution, eliminating the need for numerical optimization. To improve convergence, we incorporate the additive Schwarz acceleration as a preconditioning step following each policy update. Numerical experiments demonstrate the effectiveness of Schwarz acceleration in improving computational efficiency.

Method: Proactive data rewriting framework that keeps correct model-generated solutions as on-policy data and rewrites incorrect ones through guided re-solving, falling back to expert demonstrations only when needed. Combines data-level alignment with lightweight importance sampling for residual mismatch.

Result: Experiments on five mathematical reasoning benchmarks show consistent and significant gains over both vanilla SFT and state-of-the-art Dynamic Fine-Tuning (DFT) approach.

Conclusion: The proposed two-stage approach effectively addresses policy gap issues in off-policy learning for SFT, providing more stable optimization and improved performance through proactive data alignment.

Abstract: Supervised fine-tuning (SFT) of large language models can be viewed as an off-policy learning problem, where expert demonstrations come from a fixed behavior policy while training aims to optimize a target policy. Importance sampling is the standard tool for correcting this distribution mismatch, but large policy gaps lead to skewed weights, high variance, and unstable optimization. Existing methods mitigate this issue with KL penalties or clipping, which passively restrict updates rather than actively reducing the gap. We propose a simple yet effective data rewriting framework that proactively shrinks the policy gap before training. For each problem, correct model-generated solutions are kept as on-policy data, while incorrect ones are rewritten through guided re-solving, falling back to expert demonstrations only when needed. This aligns the training distribution with the target policy, reducing variance and improving stability. To handle residual mismatch after rewriting, we additionally apply importance sampling during training, forming a two-stage approach that combines data-level alignment with lightweight optimization-level correction. Experiments on five mathematical reasoning benchmarks show consistent and significant gains over both vanilla SFT and the state-of-the-art Dynamic Fine-Tuning (DFT) approach. Data and code will be released at https://github.com/NKU-HLT/Off-Policy-SFT.

Method: PVPO uses a reference model to pre-sample data and calculate reward scores as a reference anchor, correcting cumulative bias from intra-group comparisons and reducing rollout dependency. It also assesses sample difficulty during pre-sampling to select high-gain data.

Result: Experiments on nine datasets across two domains show PVPO achieves SOTA performance with robust generalization across tasks and scalable performance across model scales.

Conclusion: PVPO effectively addresses limitations of critic-free RL methods, is orthogonal to other advanced algorithms, and demonstrates superior efficiency and performance.

Abstract: Critic-free reinforcement learning methods, particularly group policies, have attracted considerable attention for their efficiency in complex tasks. However, these methods rely heavily on multiple sampling and comparisons within the policy to estimate advantage, which may cause the policy to fall into local optimum and increase computational cost. To address these issues, we propose PVPO, an efficient reinforcement learning method enhanced by an advantage reference anchor and data pre-sampling. Specifically, we use the reference model to rollout in advance and employ the calculated reward score as a reference anchor. Our approach effectively corrects the cumulative bias introduced by intra-group comparisons and significantly reduces reliance on the number of rollouts during training. Meanwhile, the reference model can assess sample difficulty during data pre-sampling, enabling effective selection of high-gain data to improve training efficiency. Moreover, PVPO is orthogonal to other advanced critic-free RL algorithms, making it compatible with and complementary to these methods. Experiments conducted on nine datasets across two domains demonstrate that PVPO achieves State-Of-The-Art (SOTA) performance. Our approach not only demonstrates robust generalization across multiple tasks, but also exhibits scalable performance across models of varying scales.

Method: Analyzed foundation models’ forecasting strategies by comparing them against a naive context parroting model that simply copies from the input context, tested across various dynamical systems including chaos, turbulence, oscillators, and ECG data.

Result: Context parroting consistently outperformed leading time-series foundation models across all tested systems while requiring minimal computational resources. Foundation models exhibited failure modes like converging to the mean when not parroting.

Conclusion: Current time-series foundation models rely heavily on parroting strategies rather than true learning, and understanding these limitations can guide future model design toward more sophisticated in-context learning approaches.

Abstract: Recent time-series foundation models exhibit strong abilities to predict physical systems. These abilities include zero-shot forecasting, in which a model forecasts future states of a system given only a short trajectory as context, without knowledge of the underlying physics. Here, we show that foundation models often forecast through a simple parroting strategy, and when they are not parroting they exhibit some shared failure modes such as converging to the mean. As a result, a naive context parroting model that copies directly from the context scores higher than leading time-series foundation models on predicting a diverse range of dynamical systems, including low-dimensional chaos, turbulence, coupled oscillators, and electrocardiograms – and at a tiny fraction of the computational cost. We draw a parallel between context parroting and induction heads, which explains recent works showing that large language models can often be repurposed for time series forecasting. Our dynamical systems perspective also ties the scaling between forecast accuracy and context length to the fractal dimension of the underlying chaotic attractor, providing insight into previously observed in-context neural scaling laws. By revealing the performance gaps and failure modes of current time-series foundation models, context parroting can guide the design of future foundation models and help identify in-context learning strategies beyond parroting.

Method: Combines quantization-aware training (down to 4 bits), hardware-aware hyperparameter search using Optuna, and automatic VHDL generation for a compact encoder-only Transformer architecture across forecasting, classification, and anomaly detection tasks.

Result: Achieves 0.033 mJ per inference with millisecond latency on AMD Spartan-7, with successful deployment evaluation on six public datasets across two embedded FPGA platforms (AMD Spartan-7 and Lattice iCE40).

Conclusion: The framework enables efficient integer-only Transformer accelerators for time-series tasks on embedded FPGAs, providing automated deployment solutions with significant energy and latency improvements.

Abstract: Transformer-based models have shown strong performance across diverse time-series tasks, but their deployment on resource-constrained devices remains challenging due to high memory and computational demand. While prior work targeting Microcontroller Units (MCUs) has explored hardware-specific optimizations, such approaches are often task-specific and limited to 8-bit fixed-point precision. Field-Programmable Gate Arrays (FPGAs) offer greater flexibility, enabling fine-grained control over data precision and architecture. However, existing FPGA-based deployments of Transformers for time-series analysis typically focus on high-density platforms with manual configuration. This paper presents a unified and fully automated deployment framework for Tiny Transformers on embedded FPGAs. Our framework supports a compact encoder-only Transformer architecture across three representative time-series tasks (forecasting, classification, and anomaly detection). It combines quantization-aware training (down to 4 bits), hardware-aware hyperparameter search using Optuna, and automatic VHDL generation for seamless deployment. We evaluate our framework on six public datasets across two embedded FPGA platforms. Results show that our framework produces integer-only, task-specific Transformer accelerators achieving as low as 0.033 mJ per inference with millisecond latency on AMD Spartan-7, while also providing insights into deployment feasibility on Lattice iCE40. All source code will be released in the GitHub repository (https://github.com/Edwina1030/TinyTransformer4TS).

Method: Introduces GyroBN with two necessary conditions (pseudo-reduction and gyroisometric gyrations) that guarantee theoretical control over sample statistics. The framework is instantiated on seven representative geometries including Grassmannian, constant curvature spaces, and correlation manifold.

Result: Experiments across seven geometries demonstrate the effectiveness of GyroBN. The framework incorporates several existing Riemannian normalization methods as special cases and works for all known gyrogroups in machine learning.

Conclusion: GyroBN provides a principled and effective batch normalization framework for Riemannian manifolds with gyro-structures, enabling better deep learning on non-Euclidean data.

Abstract: Normalization layers are crucial for deep learning, but their Euclidean formulations are inadequate for data on manifolds. On the other hand, many Riemannian manifolds in machine learning admit gyro-structures, enabling principled extensions of Euclidean neural networks to non-Euclidean domains. Inspired by this, we introduce GyroBN, a principled Riemannian batch normalization framework for gyrogroups. We establish two necessary conditions, namely \emph{pseudo-reduction} and \emph{gyroisometric gyrations}, that guarantee GyroBN with theoretical control over sample statistics, and show that these conditions hold for all known gyrogroups in machine learning. Our framework also incorporates several existing Riemannian normalization methods as special cases. We further instantiate GyroBN on seven representative geometries, including the Grassmannian, five constant curvature spaces, and the correlation manifold, and derive novel gyro and Riemannian structures to enable these instantiations. Experiments across these geometries demonstrate the effectiveness of GyroBN. The code is available at https://github.com/GitZH-Chen/GyroBN.git.

Method: Channel Imposed Fusion (CIF) enhances signal-to-noise ratio through cross-channel fusion, integrated with Temporal Convolutional Network (TCN) for explicit classification framework.

Result: Outperforms existing SOTA approaches on multiple EEG and ECG datasets across various classification metrics while significantly enhancing classification transparency.

Conclusion: Provides a novel perspective for medical time series classification by balancing performance with structural transparency, making it more suitable for clinical applications.

Abstract: The automatic classification of medical time series signals, such as electroencephalogram (EEG) and electrocardiogram (ECG), plays a pivotal role in clinical decision support and early detection of diseases. Although Transformer based models have achieved notable performance by implicitly modeling temporal dependencies through self-attention mechanisms, their inherently complex architectures and opaque reasoning processes undermine their trustworthiness in high stakes clinical settings. In response to these limitations, this study shifts focus toward a modeling paradigm that emphasizes structural transparency, aligning more closely with the intrinsic characteristics of medical data. We propose a novel method, Channel Imposed Fusion (CIF), which enhances the signal-to-noise ratio through cross-channel information fusion, effectively reduces redundancy, and improves classification performance. Furthermore, we integrate CIF with the Temporal Convolutional Network (TCN), known for its structural simplicity and controllable receptive field, to construct an efficient and explicit classification framework. Experimental results on multiple publicly available EEG and ECG datasets demonstrate that the proposed method not only outperforms existing state-of-the-art (SOTA) approaches in terms of various classification metrics, but also significantly enhances the transparency of the classification process, offering a novel perspective for medical time series classification.

Method: A two-stage framework: 1) Pre-training stage trains an edge masker on small labeled subset to capture key structural semantics; 2) SSL stage uses structure-aware augmentation guided by extracted structural priors to learn semantically meaningful representations.

Result: SAM-BG outperforms state-of-the-art methods on two real-world psychiatric datasets, particularly in small-labeled data settings, and uncovers clinically relevant connectivity patterns that enhance interpretability.

Conclusion: The proposed SAM-BG framework effectively preserves structural semantics in brain graphs, leading to more accurate and interpretable psychiatric diagnoses while being particularly effective in data-scarce scenarios.

Abstract: The limited availability of labeled brain network data makes it challenging to achieve accurate and interpretable psychiatric diagnoses. While self-supervised learning (SSL) offers a promising solution, existing methods often rely on augmentation strategies that can disrupt crucial structural semantics in brain graphs. To address this, we propose SAM-BG, a two-stage framework for learning brain graph representations with structural semantic preservation. In the pre-training stage, an edge masker is trained on a small labeled subset to capture key structural semantics. In the SSL stage, the extracted structural priors guide a structure-aware augmentation process, enabling the model to learn more semantically meaningful and robust representations. Experiments on two real-world psychiatric datasets demonstrate that SAM-BG outperforms state-of-the-art methods, particularly in small-labeled data settings, and uncovers clinically relevant connectivity patterns that enhance interpretability. Our code is available at https://github.com/mjliu99/SAM-BG.

Method: TESSERA uses two encoders that combine optical data with synthetic aperture radar backscatter coefficients at 10m resolution. The embeddings are fused with a multilayer perceptron to generate annual global embedding maps.

Result: TESSERA closely matches or outperforms state-of-the-art task-specific models and other foundation models across five diverse downstream tasks. It provides precomputed outputs with global, annual coverage at 10m resolution.

Conclusion: TESSERA is unprecedented in ease of use, scale, and accuracy, offering a foundation model that addresses EO data quality issues while preserving temporal signals and requiring minimal labeled data for downstream applications.

Abstract: Petabytes of satellite Earth Observation (EO) data are freely available and can address critical global challenges. However, EO data quality is poor due to clouds and variable lighting conditions. To address this, practitioners typically use compositing, but this critically removes the temporal phenological signal. Moreover, supervised machine learning to map composited pixels to task-specific classes requires accurately labelled data that are rarely available. We present TESSERA, a pixel-oriented foundation model for EO time series that creates 128-dimensional latent embeddings requiring only a few labels for task-specific training to achieve state-of-the-art performance across diverse complex tasks. TESSERA uses two encoders that combine optical data with synthetic aperture radar backscatter coefficients at 10m resolution, creating embeddings fused with a multilayer perceptron to generate annual global embedding maps. TESSERA closely matches or outperforms state-of-the-art task-specific models and other foundation models across five diverse downstream tasks. It is unprecedented in ease of use, scale, and accuracy: no other open foundation model provides precomputed outputs with global, annual coverage at 10m resolution.

Method: The authors provide both asymptotic and non-asymptotic convergence results for mini-batch QHM with increasing batch size. They analyze the trade-offs between decaying learning rates and increasing batch sizes for achieving convergence.

Result: The paper demonstrates that achieving asymptotic convergence requires either decaying learning rate or increasing batch size. However, since decaying learning rates adversely affect non-asymptotic convergence, using mini-batch QHM with increasing batch size (without decaying learning rate) proves more effective. Experiments show even finite batch size increases benefit neural network training.

Conclusion: Increasing batch size without decaying learning rate is a more effective strategy for momentum methods in stochastic nonconvex optimization, providing both theoretical convergence guarantees and practical benefits for neural network training.

Abstract: Momentum methods were originally introduced for their superiority to stochastic gradient descent (SGD) in deterministic settings with convex objective functions. However, despite their widespread application to deep neural networks – a representative case of stochastic nonconvex optimization – the theoretical justification for their effectiveness in such settings remains limited. Quasi-hyperbolic momentum (QHM) is an algorithm that generalizes various momentum methods and has been studied to better understand the class of momentum-based algorithms as a whole. In this paper, we provide both asymptotic and non-asymptotic convergence results for mini-batch QHM with an increasing batch size. We show that achieving asymptotic convergence requires either a decaying learning rate or an increasing batch size. Since a decaying learning rate adversely affects non-asymptotic convergence, we demonstrate that using mini-batch QHM with an increasing batch size – without decaying the learning rate – can be a more effective strategy. Our experiments show that even a finite increase in batch size can provide benefits for training neural networks.

Method: Extends structured state space models (SSMs) with two key mechanisms: adaptive damping for stable learning by localizing receptive fields, and learnable frequency modulation for data-driven spectral selection. Includes a factorized variant for scalable 2D problems.

Result: Achieves state-of-the-art performance on diverse PDE benchmarks (1D Burgers’, Kuramoto-Sivashinsky, 2D Navier-Stokes, compressible Euler flows) with significantly fewer parameters than competing approaches.

Conclusion: SS-NO demonstrates that damping and frequency learning are effective for operator modeling, and lightweight factorization provides a complementary path toward efficient large-scale PDE learning.

Abstract: We propose the State Space Neural Operator (SS-NO), a compact architecture for learning solution operators of time-dependent partial differential equations (PDEs). Our formulation extends structured state space models (SSMs) to joint spatiotemporal modeling, introducing two key mechanisms: adaptive damping, which stabilizes learning by localizing receptive fields, and learnable frequency modulation, which enables data-driven spectral selection. These components provide a unified framework for capturing long-range dependencies with parameter efficiency. Theoretically, we establish connections between SSMs and neural operators, proving a universality theorem for convolutional architectures with full field-of-view. Empirically, SS-NO achieves state-of-the-art performance across diverse PDE benchmarks-including 1D Burgers’ and Kuramoto-Sivashinsky equations, and 2D Navier-Stokes and compressible Euler flows-while using significantly fewer parameters than competing approaches. A factorized variant of SS-NO further demonstrates scalable performance on challenging 2D problems. Our results highlight the effectiveness of damping and frequency learning in operator modeling, while showing that lightweight factorization provides a complementary path toward efficient large-scale PDE learning.

Method: Proposes Tree-Guided Preference Optimization (TGPO) with tree-structured trajectory representation to merge semantically identical states, Process Reward Model for automatic fine-grained rewards, and dynamic weighting for prioritizing high-impact decisions.

Result: Experiments on Online-Mind2Web and C-WebShop datasets show TGPO significantly outperforms existing methods with higher success rates and fewer redundant steps.

Conclusion: TGPO effectively addresses key RL challenges for web agents through innovative trajectory representation and reward modeling, demonstrating superior performance in automated web interaction tasks.

Abstract: With the rapid advancement of large language models and vision-language models, employing large models as Web Agents has become essential for automated web interaction. However, training Web Agents with reinforcement learning faces critical challenges including credit assignment misallocation, prohibitively high annotation costs, and reward sparsity. To address these issues, we propose Tree-Guided Preference Optimization (TGPO), an offline reinforcement learning framework that proposes a tree-structured trajectory representation merging semantically identical states across trajectories to eliminate label conflicts. Our framework incorporates a Process Reward Model that automatically generates fine-grained rewards through subgoal progress, redundancy detection, and action verification. Additionally, a dynamic weighting mechanism prioritizes high-impact decision points during training. Experiments on Online-Mind2Web and our self-constructed C-WebShop datasets demonstrate that TGPO significantly outperforms existing methods, achieving higher success rates with fewer redundant steps.

Method: Evaluated predictive performance of small random feature subsets (0.02-1%) compared to full feature sets and feature selection methods across 28 out of 30 diverse datasets including microarray, RNA-Seq, mass spectrometry, and imaging data.

Result: Random feature subsets matched or outperformed both full feature sets and feature selection methods in most cases, with surprisingly low variance in results.

Conclusion: The findings challenge the reliability of computationally selected features for capturing meaningful signals and emphasize the need for rigorous validation before interpreting selected features as actionable, especially in computational genomics.

Abstract: Feature selection (FS) is assumed to improve predictive performance and identify meaningful features in high-dimensional datasets. Surprisingly, small random subsets of features (0.02-1%) match or outperform the predictive performance of both full feature sets and FS across 28 out of 30 diverse datasets (microarray, bulk and single-cell RNA-Seq, mass spectrometry, imaging, etc.). In short, any arbitrary set of features is as good as any other (with surprisingly low variance in results) - so how can a particular set of selected features be “important” if they perform no better than an arbitrary set? These results challenge the assumption that computationally selected features reliably capture meaningful signals, emphasizing the importance of rigorous validation before interpreting selected features as actionable, particularly in computational genomics.

Method: Introduces PAC-EIG, an acquisition function that maximizes information gain about the regret of the apprentice policy to efficiently identify states requiring further demonstration. Also presents Reward-EIG for reward learning objectives.

Result: Proves convergence bounds for finite state-action spaces, demonstrates failure modes of prior heuristic methods, and shows experimental advantages of the proposed approach.

Conclusion: PAC-EIG provides the first theoretical guarantee for active IRL with noisy expert demonstrations, enabling reliable policy learning with formal guarantees while minimizing demonstration costs.

Abstract: As AI systems become increasingly autonomous, reliably aligning their decision-making with human preferences is essential. Inverse reinforcement learning (IRL) offers a promising approach to infer preferences from demonstrations. These preferences can then be used to produce an apprentice policy that performs well on the demonstrated task. However, in domains like autonomous driving or robotics, where errors can have serious consequences, we need not just good average performance but reliable policies with formal guarantees – yet obtaining sufficient human demonstrations for reliability guarantees can be costly. Active IRL addresses this challenge by strategically selecting the most informative scenarios for human demonstration. We introduce PAC-EIG, an information-theoretic acquisition function that directly targets probably-approximately-correct (PAC) guarantees for the learned policy – providing the first such theoretical guarantee for active IRL with noisy expert demonstrations. Our method maximises information gain about the regret of the apprentice policy, efficiently identifying states requiring further demonstration. We also present Reward-EIG as an alternative when learning the reward itself is the primary objective. Focusing on finite state-action spaces, we prove convergence bounds, illustrate failure modes of prior heuristic methods, and demonstrate our method’s advantages experimentally.

Method: Proposes Dual Pyramid Attention Network with Temporal Pyramid (progressive downsampling) and Frequency Pyramid (band-pass filtering), connected via Cross-Pyramid Fusion Blocks using cross-attention for hierarchical information exchange.

Result: Extensive experiments show DPANet achieves state-of-the-art performance, significantly outperforming prior models on public benchmarks.

Conclusion: DPANet effectively addresses LTSF challenges by explicitly decoupling and concurrently modeling temporal multi-scale dynamics and spectral multi-resolution periodicities through dual pyramid architecture.

Abstract: Long-term time series forecasting (LTSF) is hampered by the challenge of modeling complex dependencies that span multiple temporal scales and frequency resolutions. Existing methods, including Transformer and MLP-based models, often struggle to capture these intertwined characteristics in a unified and structured manner. We propose the Dual Pyramid Attention Network (DPANet), a novel architecture that explicitly decouples and concurrently models temporal multi-scale dynamics and spectral multi-resolution periodicities. DPANet constructs two parallel pyramids: a Temporal Pyramid built on progressive downsampling, and a Frequency Pyramid built on band-pass filtering. The core of our model is the Cross-Pyramid Fusion Block, which facilitates deep, interactive information exchange between corresponding pyramid levels via cross-attention. This fusion proceeds in a coarse-to-fine hierarchy, enabling global context to guide local representation learning. Extensive experiments on public benchmarks show that DPANet achieves state-of-the-art performance, significantly outperforming prior models. Code is available at https://github.com/hit636/DPANet.

Method: Proposes a query-and-correct paradigm where the draft model populates a shared cache with candidate tokens while the target model concurrently refines the draft’s trajectory, enabling parallel processing.

Result: Achieves up to 4.83x acceleration over vanilla autoregressive decoding, significantly outperforming existing state-of-the-art methods with no fine-tuning required.

Conclusion: CARD’s cache-assisted parallel approach effectively overcomes limitations of traditional speculative decoding, enabling efficient LLM inference acceleration while maintaining model quality.

Abstract: Speculative decoding (SD), where a draft model provides multiple candidate tokens for the target model to verify in parallel, has demonstrated significant potential for accelerating LLM inference. Yet, existing SD approaches adhere to a strict draft-then-verify paradigm, enforcing a sequential process that hampers performance and constrains the draft model’s capacity. Moreover, rejecting a token in the candidate sequence invalidates all subsequent tokens, leading to wasted computation during drafting. To overcome these limitations, we propose a cache-assisted parallel speculative decoding framework called CARD, which employs a novel query-and-correct paradigm. Our approach decouples drafting from verification: the draft model populates a shared cache with candidate tokens, while the target model concurrently refines the draft’s trajectory. This enables inference at near-draft-speed, effectively leveraging the draft model’s efficiency without additional fine-tuning. Experimental results show that CARD significantly outperforms existing state-of-the-art methods, achieving up to a 4.83x acceleration over vanilla autoregressive decoding, with no fine-tuning required for either models.

Method: Aggregates attention scores to estimate token importance, selects head-specific tokens independently, aligns them into composite tokens respecting uniform cache structure, and uses global allocation to adapt retention budgets across layers.

Result: Achieves significant memory reduction while preserving accuracy, consistently outperforming prior structured and semi-structured methods.

Conclusion: The approach provides a practical and scalable solution for efficient long-context LLM deployment that remains fully compatible with standard inference pipelines.

Abstract: Large language models (LLMs) rely on key-value (KV) caches for efficient autoregressive decoding; however, cache size grows linearly with context length and model depth, becoming a major bottleneck in long-context inference. Prior KV cache compression methods either enforce rigid heuristics, disrupt tensor layouts with per-attention-head variability, or require specialized compute kernels. We propose a simple, yet effective, KV cache compression framework based on attention-guided, layer-adaptive composite tokens. Our method aggregates attention scores to estimate token importance, selects head-specific tokens independently, and aligns them into composite tokens that respect the uniform cache structure required by existing inference engines. A global allocation mechanism further adapts retention budgets across layers, assigning more capacity to layers with informative tokens. This approach achieves significant memory reduction while preserving accuracy, consistently outperforming prior structured and semi-structured methods. Crucially, our approach remains fully compatible with standard inference pipelines, offering a practical and scalable solution for efficient long-context LLM deployment.

Method: The paper uses analytical argumentation to identify four interconnected challenges: community dysfunction, research priorities misaligned with upstream needs, data fragmentation, and infrastructure inequities, tracing their roots to cultural and organizational practices.

Result: The analysis reveals that narratives deferring progress to speculative “AI scientists,” undervaluing data/infrastructure contributions, misaligned incentives, and gaps between domain experts and ML researchers constrain AI’s scientific impact.

Conclusion: Addressing these challenges requires reframing AI for science as a collective social project, emphasizing intentional community-building, cross-disciplinary education, shared benchmarks, and accessible infrastructure as prerequisites for technical progress.

Abstract: Artificial intelligence promises to accelerate scientific discovery, yet its benefits remain unevenly distributed. While technical obstacles such as scarce data, fragmented standards, and unequal access to computation are significant, we argue that the primary barriers are social and institutional. Narratives that defer progress to speculative “AI scientists,” the undervaluing of data and infrastructure contributions, misaligned incentives, and gaps between domain experts and machine learning researchers all constrain impact. We highlight four interconnected challenges: community dysfunction, research priorities misaligned with upstream needs, data fragmentation, and infrastructure inequities. We argue that their roots lie in cultural and organizational practices. Addressing them requires not only technical innovation but also intentional community-building, cross-disciplinary education, shared benchmarks, and accessible infrastructure. We call for reframing AI for science as a collective social project, where sustainable collaboration and equitable participation are treated as prerequisites for technical progress.

Method: Proposes MoT framework with: (1) Information Tinker using edge-wise semantic fusion and mixture-of-codebooks with domain-aware routing to improve information capacity; (2) Regularization Tinker with two additional regularizations to enhance gradient supervision.

Result: Experiments on 22 datasets across 6 domains show MoT achieves significant improvements in supervised, few-shot, and zero-shot scenarios compared to state-of-the-art baselines.

Conclusion: MoT effectively addresses the optimization dilemma in GFMs by tackling information bottleneck and regularization deficit issues, while maintaining scalability and achieving superior cross-domain generalization performance.

Abstract: Inspired by the success of LLMs, GFMs are designed to learn the optimal embedding functions from multi-domain text-attributed graphs for the downstream cross-task generalization capability. Among the diverse architectures, graph VQ-MAE stands out among the increasingly diverse landscape of GFM. This is attributed to its ability to jointly encode topology and textual attributes from multiple domains into discrete embedding spaces with clear semantic boundaries. Despite its potential, domain generalization conflicts cause imperceptible pitfalls. In this paper, we instantiate two of them, and they are just like two sides of the same GFM optimization coin - Side 1 Model Degradation: The encoder and codebook fail to capture the diversity of inputs; Side 2 Representation Collapse: The hidden embedding and codebook vector fail to preserve semantic separability due to constraints from narrow representation subspaces. These two pitfalls (sides) collectively impair the decoder and generate the low-quality reconstructed supervision, causing the GFM optimization dilemma during pre-training (coin). Through empirical investigation, we attribute the above challenges to Information Bottleneck and Regularization Deficit. To address them, we propose MoT - (1) Information Tinker for Two Pitfalls, which utilizes an edge-wise semantic fusion strategy and a mixture-of-codebooks with domain-aware routing to improve information capacity. (2) Regularization Tinker for Optimization Coin, which utilizes two additional regularizations to further improve gradient supervision in our proposed Information Tinker. Notably, as a flexible architecture, MoT adheres to the scaling laws of GFM, offering a controllable model scale. Compared to SOTA baselines, experiments on 22 datasets across 6 domains demonstrate that MoT achieves significant improvements in supervised, few-shot, and zero-shot scenarios.

Method: MAESTRO synergistically integrates advanced spectro-temporal modeling with multi-modal data fusion, adaptively weighting heterogeneous data sources and decomposing complex time series patterns.

Result: Evaluated on over 11 years of Hong Kong influenza data, MAESTRO demonstrates state-of-the-art performance with superior model fit (R-square of 0.956). Extensive ablations confirm significant contributions of multi-modal and spectro-temporal components.

Conclusion: The modular and reproducible pipeline is publicly available, presenting a powerful tool for epidemiological forecasting that can be deployed and extended to other regions and pathogens.

Abstract: Timely and robust influenza incidence forecasting is critical for public health decision-making. This paper presents MAESTRO (Multi-modal Adaptive Estimation for Temporal Respiratory Disease Outbreak), a novel, unified framework that synergistically integrates advanced spectro-temporal modeling with multi-modal data fusion, including surveillance, web search trends, and meteorological data. By adaptively weighting heterogeneous data sources and decomposing complex time series patterns, the model achieves robust and accurate forecasts. Evaluated on over 11 years of Hong Kong influenza data (excluding the COVID-19 period), MAESTRO demonstrates state-of-the-art performance, achieving a superior model fit with an R-square of 0.956. Extensive ablations confirm the significant contributions of its multi-modal and spectro-temporal components. The modular and reproducible pipeline is made publicly available to facilitate deployment and extension to other regions and pathogens, presenting a powerful tool for epidemiological forecasting.

Method: Formulate client selection as a restless multi-armed bandit problem and propose WILF-Q, which uses Q-learning to adaptively learn and update approximated Whittle indices for each client, then selects clients with highest indices.

Result: WILF-Q significantly outperforms existing baseline policies in terms of learning efficiency, demonstrating robust and efficient client selection in wireless FL settings.

Conclusion: WILF-Q provides a practical and effective solution for client selection in wireless FL that works without explicit knowledge of client state transitions or data distributions, making it suitable for real-world deployment.

Abstract: We consider the client selection problem in wireless Federated Learning (FL), with the objective of reducing the total required time to achieve a certain level of learning accuracy. Since the server cannot observe the clients’ dynamic states that can change their computation and communication efficiency, we formulate client selection as a restless multi-armed bandit problem. We propose a scalable and efficient approach called the Whittle Index Learning in Federated Q-learning (WILF-Q), which uses Q-learning to adaptively learn and update an approximated Whittle index associated with each client, and then selects the clients with the highest indices. Compared to existing approaches, WILF-Q does not require explicit knowledge of client state transitions or data distributions, making it well-suited for deployment in practical FL settings. Experiment results demonstrate that WILF-Q significantly outperforms existing baseline policies in terms of learning efficiency, providing a robust and efficient approach to client selection in wireless FL.

Method: Builds on momentum-based variance reduced technique and randomized Singular Value Decomposition (SVD) to create a more efficient optimizer for matrix-structured parameters in large models.

Result: LiMuon has lower memory than current Muon variants and achieves O(ε⁻³) sample complexity for non-convex stochastic optimization under both smooth and generalized smooth conditions. Experimental results on DistilGPT2 and ViT models verify efficiency.

Conclusion: LiMuon optimizer provides an efficient solution for training large models with reduced memory requirements and proven theoretical guarantees under realistic conditions that apply to modern AI tasks like LLM training.

Abstract: Large models recently are widely applied in artificial intelligence, so efficient training of large models has received widespread attention. More recently, a useful Muon optimizer is specifically designed for matrix-structured parameters of large models. Although some works have begun to studying Muon optimizer, the existing Muon and its variants still suffer from high sample complexity or high memory for large models. To fill this gap, we propose a light and fast Muon (LiMuon) optimizer for training large models, which builds on the momentum-based variance reduced technique and randomized Singular Value Decomposition (SVD). Our LiMuon optimizer has a lower memory than the current Muon and its variants. Moreover, we prove that our LiMuon has a lower sample complexity of $O(\epsilon^{-3})$ for finding an $\epsilon$-stationary solution of non-convex stochastic optimization under the smooth condition. Recently, the existing convergence analysis of Muon optimizer mainly relies on the strict Lipschitz smooth assumption, while some artificial intelligence tasks such as training large language models (LLMs) do not satisfy this condition. We also proved that our LiMuon optimizer has a sample complexity of $O(\epsilon^{-3})$ under the generalized smooth condition. Numerical experimental results on training DistilGPT2 and ViT models verify efficiency of our LiMuon optimizer.

Method: Developed a tool that uses Alternating-Time Temporal Logic (ATL) to automatically generate BDI plans that account for possible competition or cooperation between agents.

Result: The tool successfully generated plans for an illustrative game requiring agent collaboration, allowing agents to successfully achieve shared goals.

Conclusion: The ATL-based approach effectively automates BDI plan generation for multi-agent systems, accommodating both competitive and cooperative scenarios.

Abstract: Belief-Desire-Intention (BDI) is a framework for modelling agents based on their beliefs, desires, and intentions. Plans are a central component of BDI agents, and define sequences of actions that an agent must undertake to achieve a certain goal. Existing approaches to plan generation often require significant manual effort, and are mainly focused on single-agent systems. As a result, in this work, we have developed a tool that automatically generates BDI plans using Alternating-Time Temporal Logic (ATL). By using ATL, the plans generated accommodate for possible competition or cooperation between the agents in the system. We demonstrate the effectiveness of the tool by generating plans for an illustrative game that requires agent collaboration to achieve a shared goal. We show that the generated plans allow the agents to successfully attain this goal.

Method: The paper proposes Dynamic Agent Grouping ECBS (DAG-ECBS), which dynamically creates and plans agent groups while ensuring each group’s solution remains bounded suboptimal within the WinC-MAPF framework.

Result: DAG-ECBS demonstrates improved scalability compared to SS-CBS and outperforms windowed ECBS without completeness guarantees, while maintaining theoretical completeness.

Conclusion: The work provides a blueprint for designing more MAPF methods that can leverage the WinC-MAPF framework with bounded suboptimal solvers, expanding the practical applicability of complete partial-path planning approaches.

Abstract: Multi-Agent Path Finding (MAPF) is the problem of finding a set of collision-free paths for a team of agents. Although several MAPF methods which solve full-horizon MAPF have completeness guarantees, very few MAPF methods that plan partial paths have completeness guarantees. Recent work introduced the Windowed Complete MAPF (WinC-MAPF) framework, which shows how windowed optimal MAPF solvers (e.g., SS-CBS) can use heuristic updates and disjoint agent groups to maintain completeness even when planning partial paths (Veerapaneni et al. 2024). A core limitation of WinC-MAPF is that they required optimal MAPF solvers. Our main contribution is to extend WinC-MAPF by showing how we can use a bounded suboptimal solver while maintaining completeness. In particular, we design Dynamic Agent Grouping ECBS (DAG-ECBS) which dynamically creates and plans agent groups while maintaining that each agent group solution is bounded suboptimal. We prove how DAG-ECBS can maintain completeness in the WinC-MAPF framework. DAG-ECBS shows improved scalability compared to SS-CBS and can outperform windowed ECBS without completeness guarantees. More broadly, our work serves as a blueprint for designing more MAPF methods that can use the WinC-MAPF framework.

Method: Frames VAI as Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), decouples the hierarchical process using Fenchel-Rockafellar transform, and reformulates the combinatorial problem as an MDP with dense rewards to enable sequential identification of vulnerable agents.

Result: Experiments show the method effectively identifies more vulnerable agents in large-scale MARL and rule-based systems, causing worse failures and learning value functions that reveal agent vulnerability.

Conclusion: The proposed decomposition approach successfully solves the VAI problem while preserving optimal solutions and reducing computational complexity in large-scale multi-agent systems.

Abstract: Partial agent failure becomes inevitable when systems scale up, making it crucial to identify the subset of agents whose compromise would most severely degrade overall performance. In this paper, we study this Vulnerable Agent Identification (VAI) problem in large-scale multi-agent reinforcement learning (MARL). We frame VAI as a Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), where the upper level involves an NP-hard combinatorial task of selecting the most vulnerable agents, and the lower level learns worst-case adversarial policies for these agents using mean-field MARL. The two problems are coupled together, making HAD-MFC difficult to solve. To solve this, we first decouple the hierarchical process by Fenchel-Rockafellar transform, resulting a regularized mean-field Bellman operator for upper level that enables independent learning at each level, thus reducing computational complexity. We then reformulate the upper-level combinatorial problem as a MDP with dense rewards from our regularized mean-field Bellman operator, enabling us to sequentially identify the most vulnerable agents by greedy and RL algorithms. This decomposition provably preserves the optimal solution of the original HAD-MFC. Experiments show our method effectively identifies more vulnerable agents in large-scale MARL and the rule-based system, fooling system into worse failures, and learns a value function that reveals the vulnerability of each agent.

Method: Constructed Role-playing-Video60k dataset (60k videos, 700k dialogues). Developed a framework with adaptive temporal sampling: dynamic profile from video frames in temporal order, and static profile from character dialogues and summary context. Proposed evaluation method with eight metrics.

Result: Experimental results demonstrate the effectiveness of the framework, showing that dynamic role profiles significantly improve role-playing agent performance.

Conclusion: The research highlights the importance of dynamic role profiles in developing more realistic and effective role-playing agents, with the video modality proving crucial for capturing human-like perceptual dynamics.

Abstract: Role-playing agents (RPAs) have attracted growing interest for their ability to simulate immersive and interactive characters. However, existing approaches primarily focus on static role profiles, overlooking the dynamic perceptual abilities inherent to humans. To bridge this gap, we introduce the concept of dynamic role profiles by incorporating video modality into RPAs. To support this, we construct Role-playing-Video60k, a large-scale, high-quality dataset comprising 60k videos and 700k corresponding dialogues. Based on this dataset, we develop a comprehensive RPA framework that combines adaptive temporal sampling with both dynamic and static role profile representations. Specifically, the dynamic profile is created by adaptively sampling video frames and feeding them to the LLM in temporal order, while the static profile consists of (1) character dialogues from training videos during fine-tuning, and (2) a summary context from the input video during inference. This joint integration enables RPAs to generate greater responses. Furthermore, we propose a robust evaluation method covering eight metrics. Experimental results demonstrate the effectiveness of our framework, highlighting the importance of dynamic role profiles in developing RPAs.

Method: The authors build a sophisticated multimodal neural network that grounds crowdsourced descriptive keywords in visual information from movie posters to enhance keyword representations for box-office prediction.

Result: The model achieves a substantial 14.5% reduction in box-office prediction error. Analysis reveals a positive relationship between copycat status and movie revenue, but this effect diminishes with increasing numbers of similar movies and content similarity.

Conclusion: The work develops advanced deep learning tools for studying the movie industry and provides valuable business insights about copycat movies’ commercial viability.

Abstract: The movie industry is associated with an elevated level of risk, which necessitates the use of automated tools to predict box-office revenue and facilitate human decision-making. In this study, we build a sophisticated multimodal neural network that predicts box offices by grounding crowdsourced descriptive keywords of each movie in the visual information of the movie posters, thereby enhancing the learned keyword representations, resulting in a substantial reduction of 14.5% in box-office prediction error. The advanced revenue prediction model enables the analysis of the commercial viability of “copycat movies,” or movies with substantial similarity to successful movies released recently. We do so by computing the influence of copycat features in box-office prediction. We find a positive relationship between copycat status and movie revenue. However, this effect diminishes when the number of similar movies and the similarity of their content increase. Overall, our work develops sophisticated deep learning tools for studying the movie industry and provides valuable business insight.

Method: Built on freely licensed tracks from Jamendo platform, automatically annotated using Qwen-Omni model to generate question-answer pairs and captions aligned with music audio.

Result: A large-scale dataset covering diverse genres, instruments, and metadata attributes with detailed statistics and highlighted potential biases (genre and gender imbalance) to guide fair evaluation.

Conclusion: Jamendo-QA serves as a scalable public benchmark that can facilitate future research in music understanding, multimodal modeling, and fair evaluation of music-oriented QA systems.

Abstract: We introduce Jamendo-QA, a large-scale dataset for Music Question Answering (Music-QA). The dataset is built on freely licensed tracks from the Jamendo platform and is automatically annotated using the Qwen-Omni model. Jamendo-QA provides question-answer pairs and captions aligned with music audio, enabling both supervised training and zero-shot evaluation. Our resource aims to fill the gap of music-specific QA datasets and foster further research in music understanding, retrieval, and generative applications. In addition to its scale, Jamendo-QA covers a diverse range of genres, instruments, and metadata attributes, allowing robust model benchmarking across varied musical contexts. We also provide detailed dataset statistics and highlight potential biases such as genre and gender imbalance to guide fair evaluation. We position Jamendo-QA as a scalable and publicly available benchmark that can facilitate future research in music understanding, multimodal modeling, and fair evaluation of music-oriented QA systems.

Method: HGDC-Fuse constructs patient-centric multi-modal heterogeneous graphs to integrate asynchronous and incomplete data, and uses a heterogeneous graph learning module with disease correlation-guided attention to learn disease-specific modality weights based on disease correlations.

Result: On the large-scale MIMIC-IV and MIMIC-CXR datasets, HGDC-Fuse significantly outperforms state-of-the-art methods.

Conclusion: The proposed HGDC-Fuse framework effectively addresses practical challenges in multi-modal medical diagnosis and demonstrates superior performance compared to existing methods.

Abstract: Multi-disease diagnosis using multi-modal data like electronic health records and medical imaging is a critical clinical task. Although existing deep learning methods have achieved initial success in this area, a significant gap persists for their real-world application. This gap arises because they often overlook unavoidable practical challenges, such as modality missingness, noise, temporal asynchrony, and evidentiary inconsistency across modalities for different diseases. To overcome these limitations, we propose HGDC-Fuse, a novel framework that constructs a patient-centric multi-modal heterogeneous graph to robustly integrate asynchronous and incomplete multi-modal data. Moreover, we design a heterogeneous graph learning module to aggregate multi-source information, featuring a disease correlation-guided attention layer that resolves the modal inconsistency issue by learning disease-specific modality weights based on disease correlations. On the large-scale MIMIC-IV and MIMIC-CXR datasets, HGDC-Fuse significantly outperforms state-of-the-art methods.

Method: Leverages encoder of pre-trained autoencoder with noise-robust learning strategy, injecting artificial noise into latent representations during training to enhance robustness. Architecture optimized for edge devices like Raspberry Pi 4.

Result: In simulation experiments on ESC-50 dataset under 15 Hz bandwidth constraint, achieved higher macro-F1 scores than conventional sound-to-light conversion approaches.

Conclusion: The proposed method effectively addresses bandwidth and noise challenges in optical acoustic event classification systems, demonstrating superior performance in constrained environments.

Abstract: In the acoustic event classification (AEC) framework that employs Blinkies, audio signals are converted into LED light emissions and subsequently captured by a single video camera. However, the 30 fps optical transmission channel conveys only about 0.2% of the normal audio bandwidth and is highly susceptible to noise. We propose a novel sound-to-light conversion method that leverages the encoder of a pre-trained autoencoder (AE) to distill compact, discriminative features from the recorded audio. To pre-train the AE, we adopt a noise-robust learning strategy in which artificial noise is injected into the encoder’s latent representations during training, thereby enhancing the model’s robustness against channel noise. The encoder architecture is specifically designed for the memory footprint of contemporary edge devices such as the Raspberry Pi 4. In a simulation experiment on the ESC-50 dataset under a stringent 15 Hz bandwidth constraint, the proposed method achieved higher macro-F1 scores than conventional sound-to-light conversion approaches.

Method: Used a dataset with synchronized audio and respiration signals to systematically annotate pause types. Evaluated deep learning models (GRU, 1D CNN-LSTM, AlexNet, VGG16), acoustic features (MFCC, MFB), and Wav2Vec2 representations across three setups: single feature, feature fusion, and two-stage detection-classification cascade under both classification and regression formulations.

Result: Achieved per-type detection accuracy up to 89% for semantic pauses, 55% for breathing pauses, 86% for combined pauses, and 73% overall. Exertion-level classification achieved 90.5% accuracy, outperforming prior work.

Conclusion: The proposed systematic approach successfully detects different pause types in post-exercise speech and classifies exertion levels with high accuracy, demonstrating the value of combining audio and respiration signals for physiological assessment.

Abstract: Post-exercise speech contains rich physiological and linguistic cues, often marked by semantic pauses, breathing pauses, and combined breathing-semantic pauses. Detecting these events enables assessment of recovery rate, lung function, and exertion-related abnormalities. However, existing works on identifying and distinguishing different types of pauses in this context are limited. In this work, building on a recently released dataset with synchronized audio and respiration signals, we provide systematic annotations of pause types. Using these annotations, we systematically conduct exploratory breathing and semantic pause detection and exertion-level classification across deep learning models (GRU, 1D CNN-LSTM, AlexNet, VGG16), acoustic features (MFCC, MFB), and layer-stratified Wav2Vec2 representations. We evaluate three setups-single feature, feature fusion, and a two-stage detection-classification cascade-under both classification and regression formulations. Results show per-type detection accuracy up to 89$%$ for semantic, 55$%$ for breathing, 86$%$ for combined pauses, and 73$%$overall, while exertion-level classification achieves 90.5$%$ accuracy, outperformin prior work.

Method: Proposes a hybrid meta-training approach using in-context learning (ICL) that allows a single model to adapt on-the-fly to individual users without requiring separate adapters. Includes example curation strategies and data efficiency analysis.

Result: Achieves 13.9% WER on Euphonia (vs 17.5% baseline) and 5.3% WER on SAP Test 1 (vs 8% from personalized adapters). Shows that 5 curated examples can match performance of 19 random examples.

Conclusion: Presents a practical, scalable solution for dysarthric speech recognition that rivals personalized models while being more efficient and requiring less storage.

Abstract: Personalizing Automatic Speech Recognition (ASR) for dysarthric speech is crucial but challenging due to training and storing of individual user adapters. We propose a hybrid meta-training method for a single model, excelling in zero-shot and few-shot on-the-fly personalization via in-context learning (ICL). Measuring Word Error Rate (WER) on state-of-the-art subsets, the model achieves 13.9% WER on Euphonia which surpasses speaker-independent baselines (17.5% WER) and rivals user-specific personalized models. On SAP Test 1, its 5.3% WER significantly bests the 8% from even personalized adapters. We also demonstrate the importance of example curation, where an oracle text-similarity method shows 5 curated examples can achieve performance similar to 19 randomly selected ones, highlighting a key area for future efficiency gains. Finally, we conduct data ablations to measure the data efficiency of this approach. This work presents a practical, scalable, and personalized solution.

Method: AFT technique that aligns temporal features of old classes to new feature spaces using selectively compressed feature space, mitigating forgetting without retaining past data.

Result: Experiments on two datasets show consistent improvements over baseline models with accuracy gains of 3.7% to 3.9%.

Conclusion: AFT effectively addresses catastrophic forgetting in continual learning for environmental sound classification without requiring data retention, making it suitable for privacy-sensitive applications.

Abstract: As sounds carry rich information, environmental sound classification (ESC) is crucial for numerous applications such as rare wild animals detection. However, our world constantly changes, asking ESC models to adapt to new sounds periodically. The major challenge here is catastrophic forgetting, where models lose the ability to recognize old sounds when learning new ones. Many methods address this using replay-based continual learning. This could be impractical in scenarios such as data privacy concerns. Exemplar-free methods are commonly used but can distort old features, leading to worse performance. To overcome such limitations, we propose an Acoustic Feature Transformation (AFT) technique that aligns the temporal features of old classes to the new space, including a selectively compressed feature space. AFT mitigates the forgetting of old knowledge without retaining past data. We conducted experiments on two datasets, showing consistent improvements over baseline models with accuracy gains of 3.7% to 3.9%.

Method: MAGENTA geometrically decomposes regression error into radial and angular components, enabling targeted penalties for rare classes and improved directional modeling within a physically interpretable vector space.

Result: MAGENTA substantially improves SELD performance on imbalanced real-world data compared to standard regression approaches.

Conclusion: The method provides a principled foundation for geometry-aware SELD objectives and effectively addresses class imbalance issues in sound event detection systems.

Abstract: Deep learning-based Sound Event Localization and Detection (SELD) systems degrade significantly on real-world, long-tailed datasets. Standard regression losses bias learning toward frequent classes, causing rare events to be systematically under-recognized. To address this challenge, we introduce MAGENTA (Magnitude And Geometry-ENhanced Training Approach), a unified loss function that counteracts this bias within a physically interpretable vector space. MAGENTA geometrically decomposes the regression error into radial and angular components, enabling targeted, rarity-aware penalties and strengthened directional modeling. Empirically, MAGENTA substantially improves SELD performance on imbalanced real-world data, providing a principled foundation for a new class of geometry-aware SELD objectives. Code is available at: https://github.com/itsjunwei/MAGENTA_ICASSP

Method: Proposes a DNN architecture with cross-band and narrow-band blocks to estimate CTF filters through noise-free reverberant speech spectra reconstruction, followed by pseudo intrusive measurement to convert CTF estimates to time-domain RIR.

Result: Experimental results show Rec-RIR achieves state-of-the-art performance in both RIR identification and acoustic parameter estimation.

Conclusion: Rec-RIR provides an effective supervised learning approach for blind RIR identification with stable training and superior performance compared to existing methods.

Abstract: Room impulse response (RIR) characterizes the complete propagation process of sound in an enclosed space. This paper presents Rec-RIR for monaural blind RIR identification. Rec-RIR is developed based on the convolutive transfer function (CTF) approximation, which models reverberation effect within narrow-band filter banks in the short-time Fourier transform (STFT) domain. Specifically, we propose a deep neural network (DNN) with cross-band and narrow-band blocks to estimate the CTF filter. The DNN is trained through reconstructing the noise-free reverberant speech spectra. This objective enables stable and straightforward supervised training. Subsequently, a pseudo intrusive measurement process is employed to convert the CTF filter estimate into time-domain RIR by simulating a common intrusive RIR measurement procedure. Experimental results demonstrate that Rec-RIR achieves state-of-the-art (SOTA) performance in both RIR identification and acoustic parameter estimation. Open-source codes are available online at https://github.com/Audio-WestlakeU/Rec-RIR.

Method: The authors propose a generalized SRP beamforming criterion that considers generic acoustic models and spatially correlated noise, derive an optimal SRP beamformer, and analyze appropriate frequency weightings to jointly exploit level and time differences.

Result: Realistic simulations with three different microphone setups and speech under various noise conditions show significant reduction in mean localization error compared to conventional SRP, with over 60% reduction in noisy conditions.

Conclusion: The proposed generalized SRP method effectively handles real acoustic scenarios by incorporating flexible acoustic models, providing substantial improvements over conventional approaches, particularly in challenging noise conditions.

Abstract: The steered response power (SRP) method is one of the most popular approaches for acoustic source localization with microphone arrays. It is often based on simplifying acoustic assumptions, such as an omnidirectional sound source in the far field of the microphone array(s), free field propagation, and spatially uncorrelated noise. In reality, however, there are many acoustic scenarios where such assumptions are violated. This paper proposes a generalization of the conventional SRP method that allows to apply generic acoustic models for localization with arbitrary microphone constellations. These models may consider, for instance, level differences in distributed microphones, the directivity of sources and receivers, or acoustic shadowing effects. Moreover, also measured acoustic transfer functions may be applied as acoustic model. We show that the delay-and-sum beamforming of the conventional SRP is not optimal for localization with generic acoustic models. To this end, we propose a generalized SRP beamforming criterion that considers generic acoustic models and spatially correlated noise, and derive an optimal SRP beamformer. Furthermore, we propose and analyze appropriate frequency weightings. Unlike the conventional SRP, the proposed method can jointly exploit observed level and time differences between the microphone signals to infer the source location. Realistic simulations of three different microphone setups with speech under various noise conditions indicate that the proposed method can significantly reduce the mean localization error compared to the conventional SRP and, in particular, a reduction of more than 60% can be archived in noisy conditions.

Method: The system uses two main components: Text-to-Timbre (TTT) model for generating role-specific timbre embeddings from text descriptions, and Context-Aware Instruct-TTS (CA-Instruct-TTS) model that synthesizes expressive speech by analyzing contextual dialogue and incorporating emotional instructions.

Result: DeepDubbing enables automated generation of multi-participant audiobooks with timbre-matched character voices and emotionally expressive narration.

Conclusion: The proposed system offers a novel solution for audiobook production by automating the entire pipeline and addressing key limitations of current TTS approaches.

Abstract: The pipeline for multi-participant audiobook production primarily consists of three stages: script analysis, character voice timbre selection, and speech synthesis. Among these, script analysis can be automated with high accuracy using NLP models, whereas character voice timbre selection still relies on manual effort. Speech synthesis uses either manual dubbing or text-to-speech (TTS). While TTS boosts efficiency, it struggles with emotional expression, intonation control, and contextual scene adaptation. To address these challenges, we propose DeepDubbing, an end-to-end automated system for multi-participant audiobook production. The system comprises two main components: a Text-to-Timbre (TTT) model and a Context-Aware Instruct-TTS (CA-Instruct-TTS) model. The TTT model generates role-specific timbre embeddings conditioned on text descriptions. The CA-Instruct-TTS model synthesizes expressive speech by analyzing contextual dialogue and incorporating fine-grained emotional instructions. This system enables the automated generation of multi-participant audiobooks with both timbre-matched character voices and emotionally expressive narration, offering a novel solution for audiobook production.

Method: Uses Dual-Path Classifier (DPC) to combine object features from source separation model with semantic representations from pretrained classifier without fine-tuning, plus Semantic Clue Encoder (SCE) to enrich semantic depth of injected clues.

Result: Achieves state-of-the-art 11.19 dB CA-SDRi and enhanced semantic fidelity on DCASE 2025 task4 evaluation set, surpassing previous top performance of 11.00 dB.

Conclusion: Integration of separator-derived features and rich semantic clues is effective for spatial semantic segmentation, demonstrating superior performance over conventional approaches.

Abstract: The spatial semantic segmentation task focuses on separating and classifying sound objects from multichannel signals. To achieve two different goals, conventional methods fine-tune a large classification model cascaded with the separation model and inject classified labels as separation clues for the next iteration step. However, such integration is not ideal, in that fine-tuning over a smaller dataset loses the diversity of large classification models, features from the source separation model are different from the inputs of the pretrained classifier, and injected one-hot class labels lack semantic depth, often leading to error propagation. To resolve these issues, we propose a Dual-Path Classifier (DPC) architecture that combines object features from a source separation model with semantic representations acquired from a pretrained classification model without fine-tuning. We also introduce a Semantic Clue Encoder (SCE) that enriches the semantic depth of injected clues. Our system achieves a state-of-the-art 11.19 dB CA-SDRi and enhanced semantic fidelity on the DCASE 2025 task4 evaluation set, surpassing the top-rank performance of 11.00 dB. These results highlight the effectiveness of integrating separator-derived features and rich semantic clues.

Method: Uses a monotonic alignment scheme with dynamic look-ahead, built around three transformers: incremental phoneme transformer, temporal transformer predicting semantic/duration tokens, and depth transformer producing acoustic tokens.

Result: Achieves 102ms initial delay on GPU (lowest among publicly available streaming TTS), matches or surpasses larger baselines on several metrics, and delivers competitive quality in both output- and full-streaming settings.

Conclusion: VoXtream demonstrates that high-quality, low-latency streaming TTS is achievable with mid-scale training data, offering practical real-time speech synthesis capabilities.

Abstract: We present VoXtream, a fully autoregressive, zero-shot streaming text-to-speech (TTS) system for real-time use that begins speaking from the first word. VoXtream directly maps incoming phonemes to audio tokens using a monotonic alignment scheme and a dynamic look-ahead that does not delay onset. Built around an incremental phoneme transformer, a temporal transformer predicting semantic and duration tokens, and a depth transformer producing acoustic tokens, VoXtream achieves, to our knowledge, the lowest initial delay among publicly available streaming TTS: 102 ms on GPU. Despite being trained on a mid-scale 9k-hour corpus, it matches or surpasses larger baselines on several metrics, while delivering competitive quality in both output- and full-streaming settings. Demo and code are available at https://herimor.github.io/voxtream.

Method: Used t-SNE for visualizing learned features and employed probing techniques to examine how audio contributes to refining feature representations, especially for visually ambiguous or under-represented visemes.

Result: Visualization revealed natural clustering driven by visual cues, which is further refined by audio presence. Audio helps refine feature representations for ambiguous visemes.

Conclusion: The findings illuminate the interplay between modalities in AVSR and suggest new strategies for leveraging visual information to improve AVSR performance.

Abstract: Audio-Visual Speech Recognition (AVSR) models have surpassed their audio-only counterparts in terms of performance. However, the interpretability of AVSR systems, particularly the role of the visual modality, remains under-explored. In this paper, we apply several interpretability techniques to examine how visemes are encoded in AV-HuBERT a state-of-the-art AVSR model. First, we use t-distributed Stochastic Neighbour Embedding (t-SNE) to visualize learned features, revealing natural clustering driven by visual cues, which is further refined by the presence of audio. Then, we employ probing to show how audio contributes to refining feature representations, particularly for visemes that are visually ambiguous or under-represented. Our findings shed light on the interplay between modalities in AVSR and could point to new strategies for leveraging visual information to improve AVSR performance.

Method: Analyzed state-of-the-art PTM representations including paralinguistic, monolingual, multilingual, and speaker recognition models in cross-corpus SER settings.

Result: TRILLsson (a paralinguistic PTM) outperformed all other models, confirming that PSP-focused PTMs perform better in cross-corpus SER.

Conclusion: PSP-focused pre-trained models should be considered in cross-corpus SER benchmarks to enhance trustworthiness and guide reliable evaluations.

Abstract: Recent benchmarks evaluating pre-trained models (PTMs) for cross-corpus speech emotion recognition (SER) have overlooked PTM pre-trained for paralinguistic speech processing (PSP), raising concerns about their reliability, since SER is inherently a paralinguistic task. We hypothesize that PSP-focused PTM will perform better in cross-corpus SER settings. To test this, we analyze state-of-the-art PTMs representations including paralinguistic, monolingual, multilingual, and speaker recognition. Our results confirm that TRILLsson (a paralinguistic PTM) outperforms others, reinforcing the need to consider PSP-focused PTMs in cross-corpus SER benchmarks. This study enhances benchmark trustworthiness and guides PTMs evaluations for reliable cross-corpus SER.

Method: Comparative analysis of MFMs vs AFMs, followed by proposing SCAR - a fusion framework with nested cross-attention and self-attention refinement for synergistic integration of foundation models.

Result: MFMs surpass AFMs for EmoFake detection, and SCAR fusion achieves state-of-the-art performance, outperforming standalone models and conventional fusion approaches.

Conclusion: Cross-modal pre-training enables MFMs to better recognize unnatural emotional shifts, and the SCAR framework provides an effective way to fuse foundation models for superior EmoFake detection performance.

Abstract: In this work, we investigate multimodal foundation models (MFMs) for EmoFake detection (EFD) and hypothesize that they will outperform audio foundation models (AFMs). MFMs due to their cross-modal pre-training, learns emotional patterns from multiple modalities, while AFMs rely only on audio. As such, MFMs can better recognize unnatural emotional shifts and inconsistencies in manipulated audio, making them more effective at distinguishing real from fake emotional expressions. To validate our hypothesis, we conduct a comprehensive comparative analysis of state-of-the-art (SOTA) MFMs (e.g. LanguageBind) alongside AFMs (e.g. WavLM). Our experiments confirm that MFMs surpass AFMs for EFD. Beyond individual foundation models (FMs) performance, we explore FMs fusion, motivated by findings in related research areas such synthetic speech detection and speech emotion recognition. To this end, we propose SCAR, a novel framework for effective fusion. SCAR introduces a nested cross-attention mechanism, where representations from FMs interact at two stages sequentially to refine information exchange. Additionally, a self-attention refinement module further enhances feature representations by reinforcing important cross-FM cues while suppressing noise. Through SCAR with synergistic fusion of MFMs, we achieve SOTA performance, surpassing both standalone FMs and conventional fusion approaches and previous works on EFD.

Method: Uses multiple sub-centers per speaker class during training instead of a single center, allowing the embedding to capture broader speaker-specific variations while maintaining classification performance.

Result: Demonstrated effectiveness on voice conversion task, showing improved naturalness and prosodic expressiveness in synthesized speech.

Conclusion: The proposed sub-center modeling approach enables better capture of speaker variations for improved speech generation quality.

Abstract: Modeling the rich prosodic variations inherent in human speech is essential for generating natural-sounding speech. While speaker embeddings are commonly used as conditioning inputs in personalized speech generation, they are typically optimized for speaker recognition, which encourages the loss of intra-speaker variation. This strategy makes them suboptimal for speech generation in terms of modeling the rich variations at the output speech distribution. In this work, we propose a novel speaker embedding network that employs multiple sub-centers per speaker class during training, instead of a single center as in conventional approaches. This sub-center modeling allows the embedding to capture a broader range of speaker-specific variations while maintaining speaker classification performance. We demonstrate the effectiveness of the proposed embeddings on a voice conversion task, showing improved naturalness and prosodic expressiveness in the synthesized speech.

Method: The study evaluates common binaural auralization methods with/without intermediate ambisonic representation, using volumetric sampling of sound pressure or pressure+velocity on spherical/cubical surfaces. A triangular perceptual test (N=19) was conducted under reverberant and anechoic conditions.

Result: All evaluated grids achieved perceptually transparent auralization under reverberant conditions. Under anechoic conditions, only high-density spherical and cubical surface grids resulted in transparent auralization.

Conclusion: The research provides validated methods for perceptually transparent binaural auralization from wave-based simulations, with all tested methods available open source in the Chalmers Auralization Toolbox.

Abstract: Contrary to geometric acoustics-based simulations where the spatial information is available in a tangible form, it is not straightforward to auralize wave-based simulations. A variety of methods have been proposed that compute the ear signals of a virtual listener with known head-related transfer functions from sampling either the sound pressure or the particle velocity (or both) of the simulated sound field. This article summarizes the most common binaural auralization methods with and without intermediate ambisonic representation of volumetrically sampled sound pressure or sound pressure and particle velocity sampled on spherical or cubical surfaces and presents a perceptual validation thereof. A triangular test ($N=19$) confirmed that all evaluated grids resulted in a perceptually transparent auralization for the three tested sound incidence angles under reverberant conditions. Under anechoic conditions, only the high-density spherical and cubical surface grids lead to transparent auralization. All tested methods are available open source in the Chalmers Auralization Toolbox that accompanies this article.

Method: Proposes Prompt-aware Mixture (PaM) that uses multiple audio encoders with different experts to extract task-specific features based on the prompt indicating different tasks.

Result: With PaM, one Speech LLM surpasses best performances of all single-encoder Speech LLMs on ASR, Speaker Number Verification, and Audio Captioning tasks, and outperforms other fusion baselines like concatenation and averaging.

Conclusion: PaM effectively enhances Speech LLM performance by leveraging task-specific audio features through prompt-aware expert selection, demonstrating superior results across multiple audio understanding tasks.

Abstract: Connecting audio encoders with large language models (LLMs) allows the LLM to perform various audio understanding tasks, such as automatic speech recognition (ASR) and audio captioning (AC). Most research focuses on training an adapter layer to generate a unified audio feature for the LLM. However, different tasks may require distinct features that emphasize either semantic or acoustic aspects, making task-specific audio features more desirable. In this paper, we propose Prompt-aware Mixture (PaM) to enhance the Speech LLM that uses multiple audio encoders. Our approach involves using different experts to extract different features based on the prompt that indicates different tasks. Experiments demonstrate that with PaM, only one Speech LLM surpasses the best performances achieved by all single-encoder Speech LLMs on ASR, Speaker Number Verification, and AC tasks. PaM also outperforms other feature fusion baselines, such as concatenation and averaging. Our code would be available at: https://github.com/shanweiqiao/PaM

Method: The approach employs a latent diffusion transformer model with robust conditioning features to enhance degraded speech while maintaining computational efficiency.

Result: DiTSE achieves state-of-the-art audio quality matching real studio-quality audio from DAPS dataset, significantly improves speaker identity preservation and content fidelity, and reduces hallucinations compared to existing enhancers.

Conclusion: DiTSE successfully addresses key challenges in speech enhancement by providing high-quality, identity-preserving enhancement that matches studio-quality audio performance.

Abstract: Real-world speech recordings suffer from degradations such as background noise and reverberation. Speech enhancement aims to mitigate these issues by generating clean high-fidelity signals. While recent generative approaches for speech enhancement have shown promising results, they still face two major challenges: (1) content hallucination, where plausible phonemes generated differ from the original utterance; and (2) inconsistency, failing to preserve speaker’s identity and paralinguistic features from the input speech. In this work, we introduce DiTSE (Diffusion Transformer for Speech Enhancement), which addresses quality issues of degraded speech in full bandwidth. Our approach employs a latent diffusion transformer model together with robust conditioning features, effectively addressing these challenges while remaining computationally efficient. Experimental results from both subjective and objective evaluations demonstrate that DiTSE achieves state-of-the-art audio quality that, for the first time, matches real studio-quality audio from the DAPS dataset. Furthermore, DiTSE significantly improves the preservation of speaker identity and content fidelity, reducing hallucinations across datasets compared to state-of-the-art enhancers. Audio samples are available at: http://hguimaraes.me/DiTSE

Method: Two strategies: P2VA-C for structured voice attributes and P2VA-O for richer style descriptions. The framework converts persona descriptions into voice attributes for text-to-speech synthesis.

Result: P2VA-C reduces Word Error Rate (WER) by 5% and improves Mean Opinion Score (MOS) by 0.33 points. The study also discovered that current LLMs embed societal biases in voice attributes during conversion.

Conclusion: P2VA is the first framework to establish a connection between persona and voice synthesis, providing insights into the challenges of building persona-voice systems while highlighting bias issues in current LLMs.

Abstract: While persona-driven large language models (LLMs) and prompt-based text-to-speech (TTS) systems have advanced significantly, a usability gap arises when users attempt to generate voices matching their desired personas from implicit descriptions. Most users lack specialized knowledge to specify detailed voice attributes, which often leads TTS systems to misinterpret their expectations. To address these gaps, we introduce Persona-to-Voice-Attribute (P2VA), the first framework enabling voice generation automatically from persona descriptions. Our approach employs two strategies: P2VA-C for structured voice attributes, and P2VA-O for richer style descriptions. Evaluation shows our P2VA-C reduces WER by 5% and improves MOS by 0.33 points. To the best of our knowledge, P2VA is the first framework to establish a connection between persona and voice synthesis. In addition, we discover that current LLMs embed societal biases in voice attributes during the conversion process. Our experiments and findings further provide insights into the challenges of building persona-voice systems.

Method: Uses Whisper-tiny as base model, introduces hybrid training strategy combining standard and aphasic speech at varying ratios, and employs GPT-4-based reference enhancement to refine noisy aphasic transcripts for better supervision quality.

Result: Fine-tuned model significantly outperforms zero-shot baseline, reducing WER on aphasic speech by over 30% while maintaining performance on standard speech.

Conclusion: The framework provides a scalable, efficient solution for real-world disordered speech recognition, demonstrating robust generalization capabilities.

Abstract: This paper proposes AS-ASR, a lightweight aphasia-specific speech recognition framework based on Whisper-tiny, tailored for low-resource deployment on edge devices. Our approach introduces a hybrid training strategy that systematically combines standard and aphasic speech at varying ratios, enabling robust generalization, and a GPT-4-based reference enhancement method that refines noisy aphasic transcripts, improving supervision quality. We conduct extensive experiments across multiple data mixing configurations and evaluation settings. Results show that our fine-tuned model significantly outperforms the zero-shot baseline, reducing WER on aphasic speech by over 30% while preserving performance on standard speech. The proposed framework offers a scalable, efficient solution for real-world disordered speech recognition.

Method: Developed a modular benchmark framework that simulates four overlap scenarios and supports both open-sourced and commercial models with customizable metrics including dialogue behaviors, latency, prosodic adaptation, and speech quality.

Result: Benchmarking five state-of-the-art agents revealed two principal strategies: repair-first rapid yielding versus continuity-first sustained flow, with scenario-dependent performance trends.

Conclusion: The open-sourced design enables extensible evaluation of full-duplex speech systems, allowing practitioners to customize assessments for specific applications and accelerate development of robust conversational agents.

Abstract: While full-duplex speech agents promise natural, low-latency human-machine interaction by concurrently processing input and output speech, overlap management remains under-evaluated. We introduce Full-Duplex-Bench v1.5, a modular, fully automated benchmark that simulates four overlap scenarios: user interruption, listener backchannel, side conversation, and ambient speech. Our framework supports both open-sourced and commercial models, offering a comprehensive, extensible metric suite – categorical dialogue behaviors, stop and response latency, prosodic adaptation, and perceived speech quality – that can be tailored to application-specific criteria. Benchmarking five state-of-the-art agents reveals two principal strategies: repair-first rapid yielding versus continuity-first sustained flow, and highlights scenario-dependent performance trends. The open-sourced design enables seamless extension with new audio assets, languages, and deployment contexts, empowering practitioners to customize and accelerate the evaluation of robust full-duplex speech systems.

Method: Survey methodology analyzing the current trends and practical utility of deep learning approaches across three key domains: super-resolution, reconstruction, and denoising in microscopy imaging.

Result: A comprehensive overview of the rapidly growing field of deep learning-based microscopy image enhancement, documenting the evolution, current trends, and practical applications in biological and materials science contexts.

Conclusion: This survey serves as a valuable resource for researchers by providing insights into the state-of-the-art deep learning methods for microscopy image enhancement and highlighting future research directions in this rapidly advancing field.

Abstract: Microscopy image enhancement plays a pivotal role in understanding the details of biological cells and materials at microscopic scales. In recent years, there has been a significant rise in the advancement of microscopy image enhancement, specifically with the help of deep learning methods. This survey paper aims to provide a snapshot of this rapidly growing state-of-the-art method, focusing on its evolution, applications, challenges, and future directions. The core discussions take place around the key domains of microscopy image enhancement of super-resolution, reconstruction, and denoising, with each domain explored in terms of its current trends and their practical utility of deep learning.

Method: The authors train Gaussian denoisers to operate in the gradient domain and develop two analysis PnP algorithms: APnP-HQS (half-quadratic splitting) and APnP-ADMM (alternating direction method of multipliers) for image reconstruction.

Result: The proposed gradient-domain analysis PnP approach achieves performance comparable to standard image-domain PnP algorithms on image deblurring and super-resolution tasks.

Conclusion: Analysis PnP formulation with gradient-domain priors provides an effective alternative to traditional image-domain PnP methods, extending TV regularization concepts to learned regularization while maintaining competitive performance.

Abstract: Plug-and-Play Priors (PnP) is a popular framework for solving imaging inverse problems by integrating learned priors in the form of denoisers trained to remove Gaussian noise from images. In standard PnP methods, the denoiser is applied directly in the image domain, serving as an implicit prior on natural images. This paper considers an alternative analysis formulation of PnP, in which the prior is imposed on a transformed representation of the image, such as its gradient. Specifically, we train a Gaussian denoiser to operate in the gradient domain, rather than on the image itself. Conceptually, this is an extension of total variation (TV) regularization to learned TV regularization. To incorporate this gradient-domain prior in image reconstruction algorithms, we develop two analysis PnP algorithms based on half-quadratic splitting (APnP-HQS) and the alternating direction method of multipliers (APnP-ADMM). We evaluate our approach on image deblurring and super-resolution, demonstrating that the analysis formulation achieves performance comparable to image-domain PnP algorithms.

Method: Proposed an adaptive focal loss function that dynamically emphasizes both hard and easy regions based on difficulty levels and annotation variability. The method dilates hard regions identified through discrepancies between expert and non-expert annotations to better handle fuzzy capsule boundaries.

Result: The proposed method achieved superior performance with a mean dice coefficient (DSC) of 0.940 and mean Hausdorff distance (HD) of 1.949 mm on the testing dataset.

Conclusion: Integrating advanced loss functions and adaptive techniques enhances prostate capsule segmentation accuracy in micro-US images, potentially improving clinical decision-making for prostate cancer diagnosis and treatment.

Abstract: Micro-ultrasound (micro-US) is a promising imaging technique for cancer detection and computer-assisted visualization. This study investigates prostate capsule segmentation using deep learning techniques from micro-US images, addressing the challenges posed by the ambiguous boundaries of the prostate capsule. Existing methods often struggle in such cases, motivating the development of a tailored approach. This study introduces an adaptive focal loss function that dynamically emphasizes both hard and easy regions, taking into account their respective difficulty levels and annotation variability. The proposed methodology has two primary strategies: integrating a standard focal loss function as a baseline to design an adaptive focal loss function for proper prostate capsule segmentation. The focal loss baseline provides a robust foundation, incorporating class balancing and focusing on examples that are difficult to classify. The adaptive focal loss offers additional flexibility, addressing the fuzzy region of the prostate capsule and annotation variability by dilating the hard regions identified through discrepancies between expert and non-expert annotations. The proposed method dynamically adjusts the segmentation model’s weights better to identify the fuzzy regions of the prostate capsule. The proposed adaptive focal loss function demonstrates superior performance, achieving a mean dice coefficient (DSC) of 0.940 and a mean Hausdorff distance (HD) of 1.949 mm in the testing dataset. These results highlight the effectiveness of integrating advanced loss functions and adaptive techniques into deep learning models. This enhances the accuracy of prostate capsule segmentation in micro-US images, offering the potential to improve clinical decision-making in prostate cancer diagnosis and treatment planning.

Method: Compared three feature types: raw video, optical flow, and six linguistically-relevant ROIs for articulator movements. Evaluated models trained independently on each representation and multi-feature combinations.

Result: Multi-feature models consistently outperformed single-feature baselines, with the lowest phoneme error rate (PER) of 0.34 obtained by combining ROI and raw video. Temporal fidelity experiments showed reliance on fine-grained articulatory dynamics, and ROI ablation studies revealed strong contributions from tongue and lips.

Conclusion: rtMRI-derived features provide both accuracy and interpretability for phoneme recognition, establishing effective strategies for leveraging articulatory data in speech processing.

Abstract: Real-time Magnetic Resonance Imaging (rtMRI) visualizes vocal tract action, offering a comprehensive window into speech articulation. However, its signals are high dimensional and noisy, hindering interpretation. We investigate compact representations of spatiotemporal articulatory dynamics for phoneme recognition from midsagittal vocal tract rtMRI videos. We compare three feature types: (1) raw video, (2) optical flow, and (3) six linguistically-relevant regions of interest (ROIs) for articulator movements. We evaluate models trained independently on each representation, as well as multi-feature combinations. Results show that multi-feature models consistently outperform single-feature baselines, with the lowest phoneme error rate (PER) of 0.34 obtained by combining ROI and raw video. Temporal fidelity experiments demonstrate a reliance on fine-grained articulatory dynamics, while ROI ablation studies reveal strong contributions from tongue and lips. Our findings highlight how rtMRI-derived features provide accuracy and interpretability, and establish strategies for leveraging articulatory data in speech processing.

Method: Proposes an uncertainty-gated deformable network incorporating deformable feature modeling in both convolution and attention modules, an Uncertainty-Gated Enhancing Module (U-GEM) for selective feature exchange based on pixel-wise uncertainty, and a Boundary-sensitive Deep Supervision Loss for improved boundary delineation.

Result: Comprehensive experiments on two clinical breast MRI datasets show superior segmentation performance compared to state-of-the-art methods.

Conclusion: The method demonstrates strong clinical potential for accurate breast tumor delineation in MRI images.

Abstract: Accurate segmentation of breast tumors in magnetic resonance images (MRI) is essential for breast cancer diagnosis, yet existing methods face challenges in capturing irregular tumor shapes and effectively integrating local and global features. To address these limitations, we propose an uncertainty-gated deformable network to leverage the complementary information from CNN and Transformers. Specifically, we incorporates deformable feature modeling into both convolution and attention modules, enabling adaptive receptive fields for irregular tumor contours. We also design an Uncertainty-Gated Enhancing Module (U-GEM) to selectively exchange complementary features between CNN and Transformer based on pixel-wise uncertainty, enhancing both local and global representations. Additionally, a Boundary-sensitive Deep Supervision Loss is introduced to further improve tumor boundary delineation. Comprehensive experiments on two clinical breast MRI datasets demonstrate that our method achieves superior segmentation performance compared with state-of-the-art methods, highlighting its clinical potential for accurate breast tumor delineation.

Method: A no-reference dual-stream network using wavelet-based global difference perception coupled with cellular quality assessment from nuclear and membrane embeddings via Aggr-RWKV module, with cross-attention fusion and multi-term losses.

Result: Achieves >92% accuracy in detecting staining, membrane, and nuclear issues across datasets, outperforms state-of-the-art NR-IQA methods on LIVEC and KonIQ datasets, and shows strong correlation with downstream cell recognition accuracy.

Conclusion: The model enables practical pre-screening of WSI regions for computational pathology by effectively predicting quality and correlating with cell recognition performance metrics.

Abstract: Reliable whole slide imaging (WSI) hinges on image quality,yet staining artefacts, defocus, and cellular degradations are common. We present DPC-QA Net, a no-reference dual-stream network that couples wavelet-based global difference perception with cellular quality assessment from nuclear and membrane embeddings via an Aggr-RWKV module. Cross-attention fusion and multi-term losses align perceptual and cellular cues. Across different datasets, our model detects staining, membrane, and nuclear issues with >92% accuracy and aligns well with usability scores; on LIVEC and KonIQ it outperforms state-of-the-art NR-IQA. A downstream study further shows strong positive correlations between predicted quality and cell recognition accuracy (e.g., nuclei PQ/Dice, membrane boundary F-score), enabling practical pre-screening of WSI regions for computational pathology.

Method: Unsupervised GAN-based approach with multi-scale adaptive generator using Wavelet Transform and dual-branch discriminator that combines difference perception feature maps with original features.

Result: Achieves state-of-the-art denoising performance on multiple biomedical microscopy datasets, particularly excelling in high-frequency information preservation, and shows compatibility with various GAN frameworks.

Conclusion: QWD-GAN is an effective quality-aware, wavelet-driven denoising model that addresses key challenges in biomedical imaging while maintaining superior performance and adaptability.

Abstract: Image denoising plays a critical role in biomedical and microscopy imaging, especially when acquiring wide-field fluorescence-stained images. This task faces challenges in multiple fronts, including limitations in image acquisition conditions, complex noise types, algorithm adaptability, and clinical application demands. Although many deep learning-based denoising techniques have demonstrated promising results, further improvements are needed in preserving image details, enhancing algorithmic efficiency, and increasing clinical interpretability. We propose an unsupervised image denoising method based on a Generative Adversarial Network (GAN) architecture. The approach introduces a multi-scale adaptive generator based on the Wavelet Transform and a dual-branch discriminator that integrates difference perception feature maps with original features. Experimental results on multiple biomedical microscopy image datasets show that the proposed model achieves state-of-the-art denoising performance, particularly excelling in the preservation of high-frequency information. Furthermore, the dual-branch discriminator is seamlessly compatible with various GAN frameworks. The proposed quality-aware, wavelet-driven GAN denoising model is termed as QWD-GAN.

Method: The study systematically evaluates how existing pre-training methods can be integrated into state-of-the-art 3D detection architectures (both CNNs and Transformers) across various tasks and datasets.

Result: Pre-training consistently improves detection performance. Reconstruction-based self-supervised pre-training outperforms supervised pre-training, while contrastive pre-training provides no clear benefit for 3D medical object detection.

Conclusion: The research demonstrates that systematic pre-training approaches can significantly advance 3D medical object detection, with reconstruction-based self-supervised methods being the most effective strategy.

Abstract: Large-scale pre-training holds the promise to advance 3D medical object detection, a crucial component of accurate computer-aided diagnosis. Yet, it remains underexplored compared to segmentation, where pre-training has already demonstrated significant benefits. Existing pre-training approaches for 3D object detection rely on 2D medical data or natural image pre-training, failing to fully leverage 3D volumetric information. In this work, we present the first systematic study of how existing pre-training methods can be integrated into state-of-the-art detection architectures, covering both CNNs and Transformers. Our results show that pre-training consistently improves detection performance across various tasks and datasets. Notably, reconstruction-based self-supervised pre-training outperforms supervised pre-training, while contrastive pre-training provides no clear benefit for 3D medical object detection. Our code is publicly available at: https://github.com/MIC-DKFZ/nnDetection-finetuning.

Method: The proposed SLaM-DiMM framework uses diffusion models to synthesize any of the four target MRI modalities from other available modalities, incorporating a dedicated coherence enhancement mechanism to maintain structural consistency across volume depth.

Result: Qualitative and quantitative evaluations on the BraTS-Lighthouse-2025 Challenge dataset show the approach effectively generates anatomically plausible and structurally consistent MRI modality images.

Conclusion: SLaM-DiMM provides a robust solution for missing modality generation in brain MRI analysis, enabling continued use of multi-modal approaches even when some modalities are unavailable in clinical settings.

Abstract: Brain MRI scans are often found in four modalities, consisting of T1-weighted with and without contrast enhancement (T1ce and T1w), T2-weighted imaging (T2w), and Flair. Leveraging complementary information from these different modalities enables models to learn richer, more discriminative features for understanding brain anatomy, which could be used in downstream tasks such as anomaly detection. However, in clinical practice, not all MRI modalities are always available due to various reasons. This makes missing modality generation a critical challenge in medical image analysis. In this paper, we propose SLaM-DiMM, a novel missing modality generation framework that harnesses the power of diffusion models to synthesize any of the four target MRI modalities from other available modalities. Our approach not only generates high-fidelity images but also ensures structural coherence across the depth of the volume through a dedicated coherence enhancement mechanism. Qualitative and quantitative evaluations on the BraTS-Lighthouse-2025 Challenge dataset demonstrate the effectiveness of the proposed approach in synthesizing anatomically plausible and structurally consistent results. Code is available at https://github.com/BheeshmSharma/SLaM-DiMM-MICCAI-BraTS-Challenge-2025.

Method: The framework integrates Multi-axis External Weight Block (MEWB) for multi-axis frequency-domain feature extraction and improved dual attention module (DA+) with depthwise separable convolutions and spatial/channel attention mechanisms into the TransUNet architecture.

Result: On the Synapse dataset, FMD-TransUNet achieved an average DSC of 81.32% and HD of 16.35 mm across eight abdominal organs, outperforming state-of-the-art methods with a 3.84% DSC improvement and 15.34 mm HD reduction compared to baseline.

Conclusion: FMD-TransUNet effectively improves abdominal multi-organ segmentation accuracy by leveraging complementary frequency-domain and spatial-domain representations, demonstrating the value of frequency-domain analysis in medical image segmentation.

Abstract: Accurate abdominal multi-organ segmentation is critical for clinical applications. Although numerous deep learning-based automatic segmentation methods have been developed, they still struggle to segment small, irregular, or anatomically complex organs. Moreover, most current methods focus on spatial-domain analysis, often overlooking the synergistic potential of frequency-domain representations. To address these limitations, we propose a novel framework named FMD-TransUNet for precise abdominal multi-organ segmentation. It innovatively integrates the Multi-axis External Weight Block (MEWB) and the improved dual attention module (DA+) into the TransUNet framework. The MEWB extracts multi-axis frequency-domain features to capture both global anatomical structures and local boundary details, providing complementary information to spatial-domain representations. The DA+ block utilizes depthwise separable convolutions and incorporates spatial and channel attention mechanisms to enhance feature fusion, reduce redundant information, and narrow the semantic gap between the encoder and decoder. Experimental validation on the Synapse dataset shows that FMD-TransUNet outperforms other recent state-of-the-art methods, achieving an average DSC of 81.32% and a HD of 16.35 mm across eight abdominal organs. Compared to the baseline model, the average DSC increased by 3.84%, and the average HD decreased by 15.34 mm. These results demonstrate the effectiveness of FMD-TransUNet in improving the accuracy of abdominal multi-organ segmentation.

Method: PRISM incorporates a measurement-conditioned diffusion model into a theoretically principled posterior sampling scheme to handle blind inverse problems.

Result: Experiments on blind image deblurring demonstrate PRISM’s superior performance over state-of-the-art baselines in both image and blur kernel recovery.

Conclusion: PRISM effectively addresses blind inverse problems by combining measurement-conditioned diffusion priors with principled posterior sampling, advancing beyond current methods that require full forward operator knowledge.

Abstract: Diffusion models are now commonly used to solve inverse problems in computational imaging. However, most diffusion-based inverse solvers require complete knowledge of the forward operator to be used. In this work, we introduce a novel probabilistic and robust inverse solver with measurement-conditioned diffusion prior (PRISM) to effectively address blind inverse problems. PRISM offers a technical advancement over current methods by incorporating a powerful measurement-conditioned diffusion model into a theoretically principled posterior sampling scheme. Experiments on blind image deblurring validate the effectiveness of the proposed method, demonstrating the superior performance of PRISM over state-of-the-art baselines in both image and blur kernel recovery.

Method: HistDiST uses a dual-conditioning strategy with Phikon-extracted morphological embeddings and VAE-encoded H&E representations. It incorporates rescaled noise schedule, v-prediction, trailing timesteps, DDIM inversion, and eta-cosine noise schedule for controlled stochasticity. Also proposes Molecular Retrieval Accuracy (MRA) metric using GigaPath embeddings.

Result: Extensive evaluations on MIST and BCI datasets show HistDiST significantly outperforms existing methods, achieving 28% improvement in MRA on H&E-to-Ki67 translation task.

Conclusion: HistDiST effectively captures true IHC semantics and demonstrates superior performance in molecular-relevant translation compared to existing approaches.

Abstract: Hematoxylin and Eosin (H&E) staining is the cornerstone of histopathology but lacks molecular specificity. While Immunohistochemistry (IHC) provides molecular insights, it is costly and complex, motivating H&E-to-IHC translation as a cost-effective alternative. Existing translation methods are mainly GAN-based, often struggling with training instability and limited structural fidelity, while diffusion-based approaches remain underexplored. We propose HistDiST, a Latent Diffusion Model (LDM) based framework for high-fidelity H&E-to-IHC translation. HistDiST introduces a dual-conditioning strategy, utilizing Phikon-extracted morphological embeddings alongside VAE-encoded H&E representations to ensure pathology-relevant context and structural consistency. To overcome brightness biases, we incorporate a rescaled noise schedule, v-prediction, and trailing timesteps, enforcing a zero-SNR condition at the final timestep. During inference, DDIM inversion preserves the morphological structure, while an eta-cosine noise schedule introduces controlled stochasticity, balancing structural consistency and molecular fidelity. Moreover, we propose Molecular Retrieval Accuracy (MRA), a novel pathology-aware metric leveraging GigaPath embeddings to assess molecular relevance. Extensive evaluations on MIST and BCI datasets demonstrate that HistDiST significantly outperforms existing methods, achieving a 28% improvement in MRA on the H&E-to-Ki67 translation task, highlighting its effectiveness in capturing true IHC semantics.

Method: Proposed Frequency Enhanced Network (FrENet) with Adaptive Frequency Positional Modulation module that dynamically adjusts frequency components based on spectral positions, and frequency domain skip connections to preserve high-frequency details.

Result: FrENet surpasses state-of-the-art deblurring methods in RAW image deblurring, achieving significantly better restoration quality while maintaining high efficiency with reduced MACs. It also performs comparably or superior to sRGB-specific methods when extended to sRGB images.

Conclusion: FrENet effectively addresses the challenges of RAW image deblurring by operating directly in the frequency domain, demonstrating superior performance and efficiency while maintaining adaptability to sRGB images.

Abstract: Image deblurring plays a crucial role in enhancing visual clarity across various applications. Although most deep learning approaches primarily focus on sRGB images, which inherently lose critical information during the image signal processing pipeline, RAW images, being unprocessed and linear, possess superior restoration potential but remain underexplored. Deblurring RAW images presents unique challenges, particularly in handling frequency-dependent blur while maintaining computational efficiency. To address these issues, we propose Frequency Enhanced Network (FrENet), a framework specifically designed for RAW-to-RAW deblurring that operates directly in the frequency domain. We introduce a novel Adaptive Frequency Positional Modulation module, which dynamically adjusts frequency components according to their spectral positions, thereby enabling precise control over the deblurring process. Additionally, frequency domain skip connections are adopted to further preserve high-frequency details. Experimental results demonstrate that FrENet surpasses state-of-the-art deblurring methods in RAW image deblurring, achieving significantly better restoration quality while maintaining high efficiency in terms of reduced MACs. Furthermore, FrENet’s adaptability enables it to be extended to sRGB images, where it delivers comparable or superior performance compared to methods specifically designed for sRGB data. The code will be available at https://github.com/WenlongJiao/FrENet .

Method: Benchmarked state-of-the-art neural network architectures, proposed novel architectures, and developed semi-supervised frameworks (DIST, SemiVT-Surge, ENCORE) that leverage unlabeled surgical video through dynamic pseudo-labeling to reduce reliance on labeled data.

Result: Achieved state-of-the-art results on challenging surgical datasets using minimal labeled data, and released two large multi-task datasets (GynSurg and Cataract-1K) to support reproducibility and advance the field.

Conclusion: This work contributes to robust, data-efficient, and clinically scalable solutions for surgical video analysis, laying the foundation for generalizable AI systems that can meaningfully impact surgical care and training.

Abstract: Advances in surgical video analysis are transforming operating rooms into intelligent, data-driven environments. Computer-assisted systems support full surgical workflow, from preoperative planning to intraoperative guidance and postoperative assessment. However, developing robust and generalizable models for surgical video understanding remains challenging due to (I) annotation scarcity, (II) spatiotemporal complexity, and (III) domain gap across procedures and institutions. This doctoral research aims to bridge the gap between deep learning-based surgical video analysis in research and its real-world clinical deployment. To address the core challenge of recognizing surgical phases, actions, and events, critical for analysis, I benchmarked state-of-the-art neural network architectures to identify the most effective designs for each task. I further improved performance by proposing novel architectures and integrating advanced modules. Given the high cost of expert annotations and the domain gap across surgical video sources, I focused on reducing reliance on labeled data. We developed semi-supervised frameworks that improve model performance across tasks by leveraging large amounts of unlabeled surgical video. We introduced novel semi-supervised frameworks, including DIST, SemiVT-Surge, and ENCORE, that achieved state-of-the-art results on challenging surgical datasets by leveraging minimal labeled data and enhancing model training through dynamic pseudo-labeling. To support reproducibility and advance the field, we released two multi-task datasets: GynSurg, the largest gynecologic laparoscopy dataset, and Cataract-1K, the largest cataract surgery video dataset. Together, this work contributes to robust, data-efficient, and clinically scalable solutions for surgical video analysis, laying the foundation for generalizable AI systems that can meaningfully impact surgical care and training.

Method: Extends FCOS with a Feedback Attention Ladder CNN for classification of normal vs abnormal mitotic figures, feeding into a fusion network that adjusts FCOS bounding box predictions.

Result: Achieved an F1 score of 0.655 for mitosis detection on the preliminary evaluation dataset.

Conclusion: The composite model successfully improves mitotic figure detection performance by combining object detection with specialized classification feedback.

Abstract: We present a novel approach which extends the existing Fully Convolutional One-Stage Object Detector (FCOS) for mitotic figure detection. Our composite model adds a Feedback Attention Ladder CNN (FAL-CNN) model for classification of normal versus abnormal mitotic figures, feeding into a fusion network that is trained to generate adjustments to bounding boxes predicted by FCOS. Our network aims to reduce the false positive rate of the FCOS object detector, to improve the accuracy of object detection and enhance the generalisability of the network. Our model achieved an F1 score of 0.655 for mitosis detection on the preliminary evaluation dataset.